Lexicon Enhancement: Support for arrays of strings in procedure and query encodings

[mikekasprzak]

TypeScript patch is here: bluesky-social/atproto#3986

Draft Go patch is here: bluesky-social/indigo#1107

---

Thus far my work on adding native ActivityPub support to the Bluesky PDS hasn't required any specification changes, but this is the first.

In short, ActivityPub has two official and one unofficially supported Content-Type header.

• application/ld+json; profile="https://www.w3.org/ns/activitystreams"
• application/activity+json
• application/json

It's not possible to correctly interoperate with other ActivityPub servers without allowing multiple input Content-Type's. The document bodies are identical (all based on JSON LD), which is what I'm proposing here.

Below is an example Lexicon document demonstrating what it would look like.

{
  "lexicon": 1,
  "id": "org.w3.activitypub.putInbox",
  "defs": {
    "main": {
      "type": "procedure",
      "input": {
        "encoding": [
          "application/json",
          "application/activity+json",
          "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\""
        ],
        "schema": { "type": "ref", "ref": "org.w3.activitypub.object" }
      },
      "output": {
        "encoding": "application/activity+json",
        "schema": { "type": "ref", "ref": "org.w3.activitypub.object" }
      }
    }
  }
}

Implementations should only ever allow 1 of the listed types.

[mikekasprzak]

Tangentially, I wanted to mention that changing the spec to always require an array of encodings (even if it's just one), might be a cleaner solution, as right now there doesn't seem to be any (many?) "multiple possibilities options" in the spec (i.e. String or String Array).

Notably, getting Lexicon to support ActivityPub required some funky changes to the lex-cli tool as many values can be a string, an object, and sometimes also an array of strings or objects. 😮. Choosing Array of Strings "always" for encodings would certainly simplify things while keeping flexibility.

[bnewbold]

Thanks for digging in to this and giving it a shot!

I've got a couple layers of feedback.

The first is on the underlying motivation and what you are trying to achieve. Being a "complete" or full-power schema language for all possible HTTP APIs isn't a design goal or aspiration for Lexicon. Representing an entire other protocol like ActivityPub using lexicon endpoint schemas feels like a stretch, and the benefit is unclear to me. I think it is totally fine for atproto network service implementations (PDS, appview, relay, etc) to support other APIs and protocols, without expressing that in lexicon. For example, the Bluesky app supports RSS, and doesn't try to express that in Lexicon.

In terms of the more specific request here around input encoding flexibility, I could see a couple options. One is that you could just use a wildcard (*/*), which is what we do for the com.atproto.repo.uploadBlob endpoint. All the encodings you give in the example would also be covered by application/*. MIME types based on JSON are popular, so we could also consider loosening the validation behaviors. For example, if an endpoint schema specifies an encoding of application/activity+json, maybe the specs should say that application/json is also allowed? Would require some more research, maybe there is already guidance or best practices around this.

In terms of actually changing the lexicon language, i'd be very resistant to either changing the type of the existing field name, or to making a lexicon field have flexible type (eg, "string or array of strings"). I think we should keep things tightly typed, and need to avoid breaking changes. If we needed to evolve this part of the language, we'd probably want to add a new field name and mark the existing one as deprecated (but still supported).