Skip to content

Move circle definitions into the Circle table - #1661

Closed
toddmitchell wants to merge 3 commits into
review-stamp-cat1from
circle-definitions-to-table
Closed

Move circle definitions into the Circle table#1661
toddmitchell wants to merge 3 commits into
review-stamp-cat1from
circle-definitions-to-table

Conversation

@toddmitchell

@toddmitchell toddmitchell commented Aug 21, 2026

Copy link
Copy Markdown
Member

Groundwork for Cat 3 (the enrollment model). Stacked on review-stamp-cat1 (#1649) — retarget to main once that lands. Independent of Cat 2 (#1655).

Why this has to happen first

Circle.AppId, GrantOn, Designation and Emoji shipped dormant with the drive-addressing schema work. Nothing was ever wired to them — in fact grep for TableCircle across src/services and src/apps returns nothing. The table has no callers at all.

Circle definitions actually live in CircleDefinitionService's ThreeKeyValueStorage blob, which is exactly what part 1 warned about:

Bury AppId in that blob and: delete app ⇒ delete its circles becomes load-all-deserialize-filter-delete… and no constraint or FK can ever reference it.

The enrollment pipeline is specified as a table query (part 2, L188-194):

The server queries the Circle tableWHERE GrantOn = Connect (this query is why GrantOn is an indexed column)… The stored registration JSON is not read here; rows are the truth.

You can't run that against a blob store. So the definitions have to move before any of Cat 3 can be built.

What changed

CircleDefinition gains AppId, GrantOn, Designation, Emoji. They're columns, and a second copy inside the row's data blob would let a query on the column disagree with the hydrated object — the same promote-don't-duplicate rule Cat 1 applied to ReviewedAt.

They are not [JsonIgnore], and the second commit explains why: this same type is the wire shape. CircleDefinitionControllerBase serves it directly and takes one as an update body, so hiding the fields from JSON would starve clients and make every client-side circle update arrive with GrantOn defaulted — silently resetting it. Data loss on an ordinary edit.

One attribute can't make one type be both the wire shape and the stored shape, so the blob copy is cleared inside ToRecord instead, the same trick ToConnectionsRecord uses for the grant collections. AppId is also no longer taken from an update request: ownership is set at creation and must not be reassignable by anyone who can PUT a definition.

Equality and GetHashCode account for all four, because EnsureSystemCirclesExistAsync reconciles definitions by comparing them — miss that and system circles silently stop reconciling.

RedactedCircleDefinition exposes the same four. Part 4 renders circles by designation and emoji.

New enums CircleGrantOn (None | Connect | OwnFlowConnect | Review) and CircleDesignation (Personal | Audience | Vendor), matching the column values.

Table layerTableCircle grows UpsertAsync and GetAllAsync; TableCircleCached wraps both and invalidates the new all-key alongside the per-circle key.

CircleDefinitionService moves off ThreeKeyValueStorage onto db.CircleCached, with ToRecord/FromRecord doing the column-vs-blob split.

v13 → v14 copies existing definitions across. Idempotent and additive — a definition already in the table is skipped rather than overwritten, so a partial run repeats safely. The blob rows are deliberately left in place: if this goes wrong the source data is still sitting there. Cleaning them up is a separate job.

Behaviour

None. All four fields take their defaults for every existing circle (AppId null, GrantOn None, Designation Personal, Emoji null), which is what they already were.

What's not here

The other part-1 leftover: AppRegistrations exists as a table but AppRegistrationService still writes to the KeyThreeValue blob. Nothing in the circle work needs it until DefaultCircles lands in the next PR, so it's cleaner as its own change than bulking this one out.

Testing

Full solution suite green: 2,524 passed, 0 failed (68 skipped, all pre-existing [Explicit]).

Not verified: the v13→v14 migration has not been run against a live PostgreSQL or SQLite tenant holding real blob-stored definitions. Given this moves where every circle definition lives, that's the check I'd want before it leaves draft.

🤖 Generated with Claude Code

Groundwork for the enrollment model. Definitions lived in the shared
key-three-value blob, which is why AppId and GrantOn could never be queried
or constrained -- the columns shipped dormant with the drive-addressing
schema work and nothing was ever wired to them. TableCircle had no caller at
all.

The enrollment pipeline has to answer "which circles enrol on connect?" on
the hot path, and that is a WHERE GrantOn = ? against an indexed column, not
a load-all-deserialize-filter over opaque rows. So the definitions have to
live in the table before any of Cat 3 can be built.

- CircleDefinition gains AppId, GrantOn, Designation and Emoji. All four are
  [JsonIgnore]: they are columns, and a second copy inside the row's data
  blob would let a query on the column disagree with the hydrated object.
  Equality and GetHashCode account for them, since EnsureSystemCirclesExist
  reconciles definitions by comparing them.
- RedactedCircleDefinition exposes the same four without [JsonIgnore] -- it
  is the wire shape, and the client proposal renders circles by designation
  and emoji.
- CircleGrantOn and CircleDesignation are new enums matching the column
  values. Every existing circle is None/Personal, so nothing changes
  behaviour until something sets them.
- TableCircle grows UpsertAsync and GetAllAsync; TableCircleCached wraps
  both and invalidates the all-key alongside the per-circle key.
- CircleDefinitionService moves off ThreeKeyValueStorage onto db.CircleCached
  with ToRecord/FromRecord doing the column-vs-blob split.
- v13 -> v14 copies existing definitions across. Idempotent and additive: a
  definition already in the table is left alone, so a partial run repeats
  safely. The blob rows are deliberately left in place -- if this goes wrong
  the source data is still there. Cleaning them up is a separate job.

No behaviour change. The four fields take their defaults for every existing
circle, which is what they already were.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
[JsonIgnore] was the wrong tool. CircleDefinition is not only the stored
shape -- CircleDefinitionControllerBase serves it directly and takes one as
an update body -- so hiding AppId, GrantOn, Designation and Emoji from JSON
starved clients of fields the client proposal renders circles by, and worse,
made every client-side circle update arrive with GrantOn defaulted and
silently reset it. Data loss on an ordinary edit.

One attribute cannot make one type be both the wire shape and the stored
shape. The fields are back on the wire, and the blob copy is cleared inside
ToRecord instead -- the same clear-before-serialize trick ToConnectionsRecord
already uses for the grant collections. The guarantee is unchanged: nothing
in the blob can drift from the column, because deserializing the blob alone
yields defaults.

AppId is also no longer taken from an update request. Ownership is set when
the circle is created and must not be reassignable by anyone who can PUT a
definition.

Tests pin both directions: no promoted value survives into the blob, the
caller's object is intact afterwards, the fields round-trip through the
record, they are visible on the wire, and an update body echoed back does
not reset GrantOn.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@toddmitchell
toddmitchell marked this pull request as ready for review August 22, 2026 00:59
@toddmitchell

Copy link
Copy Markdown
Member Author

Superseded by #1688, which replays this onto main (migration renumbered to v12->v13, folded together with the app-registrations move). Branch deleted; every commit on it is reachable from integration-connection-defaults.

@toddmitchell
toddmitchell deleted the circle-definitions-to-table branch August 26, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant