Open discussion on query-json language design decisions

[davesnx]

We're opening this issue to gather community feedback on the language design decisions we've made for query-json v1. While we've put a lot of thought into these choices, we want to hear other views from our users.

Context

query-json started as a jq-compatible implementation but evolved into it's own different language. We made intentional breaking changes where we believed jq's design was confusing or could be improved. These decisions are documented in our jq Compatibility Guide.

Key design decisions

1. Naming conventions

• snake_case everywhere (to_string instead of tostring)
• More descriptive names (to_uppercase instead of ascii_upcase, trim_start instead of ltrimstr)
• fn instead of def for user-defined functions
• trim_left instead of ltrimstr and trim_right instead of rtrimstr, so autocompletion in the repl and user discovery is better
• byte_length instead of utf8bytelength

Question: Are there any renamed functions that feel awkward or harder to discover?

2. Behavioral changes

• group_by returns an object instead of nested arrays
• keys and unique preserve insertion order (instead of sorting)
• infinite is a generator instead of IEEE infinity

Question: Do any of these changes cause friction in your workflows?

3. Optional access on functions

We extended ? to work on any expression, not just field access: first?, keys?, nth(10)?.

Question: Is this intuitive? Any edge cases we should consider?

What should we do with unsupported functions?

Some jq functions are not implemented. Currently, using them produces errors like:

Function	Current behavior
strftime, strptime	error: time formatting not implemented
modulemeta	error: modulemeta not implemented
tojsonstream, fromjsonstream, truncate_stream	error: JSON stream functions not implemented
tojson, fromjson	error: not implemented, use to_string (input is already JSON)
input_filename, input_line_number	error: input metadata not implemented
Modules (import, include)	Not implemented
Format strings (@base64, @uri, @csv, etc.)	Not implemented

Questions:

1. Should we prioritize implementing any of these?
2. For functions we don't plan to implement, should we provide better error messages with workarounds?
3. Are there jq functions you rely on that aren't listed here?
4. Any other thing you hate/love about jq?
5. Are the added functions worth it? https://github.com/davesnx/query-json/blob/main/docs/JQ_COMPATIBILITY.md#additional-features-in-query-json
6. Should we implement modules differently from jq and have "namespaces" for groups of functions? I'm thinking Math.sin, Math.pi, etc instead of being at the root namespace?

[ctrlok]

Optional access on functions is partly very good idea, still it makes some advanced requests pretty questionable :badumts: 🥁

As an example for json

{
  "users": [
    { "id": 1, "name": "alice", "active": true },
    {
      "id": 2,
      "name": "bob",
      "metadata": {
        "preferences": [
          "dark_mode",
          { "notifications": { "email": "bob@example.com" } }
        ]
      }
    }
  ]
}

the access string will look like

.users[].metadata?.preferences?[]?.notifications?.email?

also, in result I'll see
null
null
bob@example.com

where first null is from 'id: 1' object and second null is from preferences key

It is not quite intuitive

Jq is not better here - it allow .users[].metadata?.preferences?[]?.notifications?.email (without question mark in the end)

alternative

I'm wondering if there is scenario for aplicative functors for traversal?

extractEmails :: Value -> [Text]
extractEmails v = v ^.. key (fromString "users") . values
                      . key (fromString "metadata")
                      . key (fromString "preferences") . values
                      . key (fromString "notifications")
                      . key (fromString "email") . _String

in query-json language it may translates to

.users[].metadata.preferences[].notifications.email

where all 'nulls' will be silently ignored.

we still can use ? if we want nulls to be shown, but then the nullable field will be more controlled

as example

.users[].metadata?.preferences[].notifications.email

will return

null
bob@example.com

but we will know that this null is created because some users[] didn't have metadata, because only metadata was questionable

also, we can use ! as must

.users[].metadata!.preferences[].notifications.email will fail, because not all users have .metadata

is this make sense? I can provide more examples and elaborate more on it if my attempts is not clear