Skip to content

Make sury-ppx read and write JSON the way Serde does - #491

Closed
rolandpeelen wants to merge 1 commit into
DZakh:mainfrom
rolandpeelen:sury-ppx-serde-compatibility
Closed

rolandpeelen wants to merge 1 commit into
DZakh:mainfrom
rolandpeelen:sury-ppx-serde-compatibility

Conversation

@rolandpeelen

Copy link
Copy Markdown

What changes for you

The ppx now generates schemas that read and write JSON the way Serde does for the matching Rust type, in its default mode. A ReScript app and a Rust service can share their types.

@schema
type shape = Circle(float) | Rect(float, float) | Labeled({label: string, size: option<float>}) | Empty

@schema
type t = {shape: shape, note: option<string>, done: unit}
// Rect(1., 2.) with note None, encoded
// before
{shape: {TAG: "Rect", _0: 1, _1: 2}, note: undefined, done: undefined}
// after: the JSON value serde_json writes for the Rust mirror
{shape: {Rect: [1, 2]}, note: null, done: null}
ReScript Rust JSON before JSON now
option<'a> Option<A> undefined for None null for None; a record field also reads a missing key as None
unit () undefined null
Ctor('a) Ctor(A) {"TAG": "Ctor", "_0": a} {"Ctor": a}
Ctor('a, 'b, 'c) Ctor(A, B, C) {"TAG": "Ctor", "_0": a, "_1": b, "_2": c} {"Ctor": [a, b, c]}
Ctor({x: 'a}) Ctor { x: A } {"TAG": "Ctor", "x": a} {"Ctor": {"x": a}}
#tag('a) tag(A) {"NAME": "tag", "VAL": a} {"tag": a}
@tag("kind") constructor without payload #[serde(tag = "kind")] unit variant "Ctor" {"kind": "Ctor"}
  • An externally tagged object holds its one key and nothing else, as serde_json demands.
  • @as on a constructor names its key, like #[serde(rename)]. A non-string @as on a constructor with a payload is a compile error.
  • @s.default still fills in only a missing value and rejects null, like #[serde(default)].
  • @s.null on a ? field makes the key required. @s.nullable writes None as a missing key, like #[serde(default, skip_serializing_if = "Option::is_none")].

The sury-ppx README gains a "Serde compatibility" section with the type and attribute mapping.

Impact

  • Breaking for ppx users. Every option, unit, constructor with a payload and polymorphic variant with a payload changes its JSON. To keep an old shape, describe it with @s.matches, for example @s.matches(S.option(S.string)) option<string>. The README says so.
  • packages/sury/src is untouched. The change is the ppx's codegen (Structure.ml), its tests and docs, and a new Rust crate.
  • Protobuf keeps working: the option<address> field in Ppx_Protobuf_test.res round-trips through S.protobuf with the new option shape.

How it's held to Serde

packages/serde-compat is a Rust crate that mirrors every @schema type in packages/e2e/src/ppx as the type Serde reads the same JSON into: 149 types, 527 cases. cargo run writes Serde's verdicts to fixtures.json:

mirror::<SimpleRecord>("simpleRecord")
    .values([SimpleRecord { label: "foo".into(), value: 1 }])    // what Serde writes, read back to itself
    .accepts(&[r#"{"value":1,"label":"foo","unknown":[1]}"#])     // what it reads but never writes
    .rejects(&[r#"{"label":"foo"}"#])                             // what it refuses
    .diverges(r#"["foo",1]"#, "Serde also reads a struct from an array of its fields in order")

Each e2e test file then holds its schemas to those verdicts:

assertSerdeCompatible("Record", [("simpleRecord", simpleRecordSchema->S.castToUnknown)])
  • Sury must read what Serde reads, refuse what it refuses, and write back the JSON Serde writes. Serde reads its own output back, so this checks both directions.
  • .accepts and .rejects assert Serde agrees, so a mirror can't record a verdict it didn't mean.
  • A .diverges case fails the suite once Sury starts agreeing, so a closed gap can't linger.
  • cargo run -- check is a new CI job and a pnpm verify step. It fails on a stale fixtures.json, or on a @schema type with no mirror or skip.
  • CLAUDE.md and CONTRIBUTING.md state the rule: a new ppx test type gets a mirror.

On main's ppx, 44 of the 149 mirrored types fail this check. On this branch all pass.

What isn't Serde yet

packages/serde-compat/README.md lists it, with a todo list for full compatibility. In short:

  • Known differences (the 6 .diverges cases):
    • Serde also reads a unit variant from {"Variant": null} and a struct from an array of its fields in order.
    • Serde refuses 1.0 for an integer and a repeated key; JSON.parse erases both before Sury sees them.
    • -0.0 comes out as 0.
    • A top-level @s.nullable writes undefined.
  • Representation limits:
    • An @unboxed constructor without a payload reads its name, where Serde's untagged unit variant is null.
    • An @tag constructor whose payload isn't an inline record keeps it under _0. Serde refuses tag on a tuple variant at compile time.
  • Not mirrored (9 types): S.url, S.date, S.blob, S.formData, S.urlSearchParams hold JS instances, plus three @s.matches schemas with no JSON form. Each skip carries its reason.
  • Todo, in rough order: rename_all, flattening an @tag newtype's struct next to the tag, adjacent tagging (tag + content), alias, other, skip*, container default, integer widths beyond i32/u16/2^53, non-string map keys, tuple and unit structs.

Verification

  • e2e: 299 of 299 tests pass, 13 files, with the ppx built from this branch.
  • cargo run --locked -- check up to date; cargo clippy and cargo fmt --check clean.
  • pnpm lint:deadcode clean.
  • The ppx builds against ppxlib 0.34 (what CI pins) and 0.35.
  • A 3-argument constructor Foo(int, int, string) was checked against serde_json directly: {"Foo":[1,2,"a"]} both ways, _0, _1, _2 landing on positions 0, 1, 2, and the same refusals for short, long and map-shaped payloads.
  • Not run locally: the full pnpm verify (it touches nothing under packages/sury), and the CI ppx build on Linux.

Follow-up

The fixtures compare JSON in with JSON out, so they can't catch two same-typed positional slots being swapped the same way in both directions. The permanent tests only have multi-argument constructors whose slots differ in type, where a swap fails the type check. A Foo(int, int, string) case with direct value assertions would close that.

The ppx now generates Serde's default representation:

- option<'a> reads and writes None as null; a record field of type
  option<_>, or one marked ?, also reads a missing key as None
- unit is null
- a constructor with a payload is externally tagged: {"Ctor": a},
  {"Ctor": [a, b]}, {"Ctor": {"x": a}}, one key and nothing else
- a polymorphic variant with a payload is {"tag": a}
- an @tag constructor without a payload is {"kind": "Ctor"}
- @as on a constructor names its key

packages/serde-compat mirrors every @Schema type in the ppx tests as
the Rust type Serde reads the same JSON into, and writes Serde's
verdicts to fixtures.json. Each e2e test file holds its schemas to them
with assertSerdeCompatible, in both directions. `cargo run -- check`
runs in CI and fails on a stale fixtures.json or an unmirrored type.

The README in packages/serde-compat lists the differences that remain
and the Serde features the ppx doesn't cover yet.
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@rolandpeelen

Copy link
Copy Markdown
Author

Never meant to open this - sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant