Skip to content

refactor(daemon): give the Maestro fallback and ambiguous-match details real types - #1612

Merged
thymikee merged 1 commit into
mainfrom
claude/type-interaction-boundaries
Aug 5, 2026
Merged

refactor(daemon): give the Maestro fallback and ambiguous-match details real types#1612
thymikee merged 1 commit into
mainfrom
claude/type-interaction-boundaries

Conversation

@thymikee

@thymikee thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member

Summary

Three places smuggled structured data through untyped bags and re-read it with runtime guards. Each gets an explicit typed boundary, so the guards and the paragraphs explaining them go away.

No user-visible behavior change except one intended fix, called out in Finding C.

A — one rule, two encodings: the Maestro coordinate-fallback resolution suppression

interaction-touch-response.ts calls itself "the single construction site … composed in exactly one place", but it encoded ONE rule two syntactically unrelated ways:

// runner-payload branch
...(source.maestroFallbackUsed ? {} : { resolution: DIRECT_IOS_NOT_OBSERVED_RESOLUTION }),

// runtime branch — destructures unconditionally to use the result conditionally, states no reason
const { resolution: _resolution, ...resultExtraWithoutResolution } = resultExtra;
...(source.maestroFallbackUsed ? resultExtraWithoutResolution : resultExtra),

Both mean "the Maestro coordinate path ran, so resolutionDisclosure is inapplicable (ADR 0012)", and the rationale lived only in a type comment on the other source variant.

Now both branches read one predicate through one helper:

...applyResolutionDisclosurePolicy(source, { resolution: DIRECT_IOS_NOT_OBSERVED_RESOLUTION })
...applyResolutionDisclosurePolicy(source, interactionResultExtra(result))

The ADR 0012 reason is stated once, at applyResolutionDisclosurePolicy, including why cell membership in maestro-non-hittable-fallback is usage-based (allowed-but-not-taken is still the direct path). The discriminator is renamed after the dispatch path that executed rather than the flag that permitted it — maestroCoordinateFallbackDispatched — and hoisted into a shared base so it is declared once instead of on both union arms.

handleFillCommand's two-arm call also collapses:

const result = await interactor.fill(x, y, text, delayMs, {
  allowNonHittableCoordinateFallback: context?.allowNonHittableCoordinateFallback === true,
});

Checked before collapsing, as asked: no Interactor implementation distinguishes "options absent" from "options present with false". platforms/apple/interactions.ts spreads the runner field only when truthy (false and undefined both omit it); core/interactors/{android,linux,web}.ts, provider-limrun, and provider-webdriver ignore the parameter entirely. Only a test mock recorded the arity, and it is updated to assert the single call shape across all three permission states. No bug to report here.

The guarantee matrix was re-read: the maestro-non-hittable-fallback resolutionDisclosure cell already documents usage-based membership, and every via pointer still names buildInteractionResponseData. The matrix still tells the truth; no cell changed. src/__tests__/contracts/interaction-guarantees.test.ts passes (8 tests).

B — two structured outputs riding an untyped Record

Interactor.type. #1588 widened it to Promise<Record<string, unknown> | void> purely so a textEntryRoute string could escape, and the caller re-derived it with typeof textEntryRoute === 'string'. It is now Promise<TypeTextBackendResult | void>, with TextEntryRoute a closed union of the four routes the Swift runner assigns. The narrowing happens once, at the Apple runner boundary (readTypeTextBackendResult) — the trust boundary AGENTS.md designates — and handleTypeCommand's guard is gone. Every non-Apple implementation already returned void.

Because the narrowing drops a route it cannot name, a Swift-side addition would otherwise silently vanish from the response. A route-parity test reads the runner sources and pins the union against every textEntryRoute literal the Swift side assigns.

maestroFallbackDetails. Returned Record<string, unknown>, and both call sites then re-read fallbackDetails.maestroNonHittableCoordinateFallbackUsed === true out of the bag they had just constructed. It now returns a typed { used, extra }: used selects the dispatch path for Finding A's predicate, extra is the typed response-field set. Neither call site re-reads anything.

The wide Record<string, unknown> | void across the rest of the Interactor surface is untouched pre-existing debt.

C — details.candidates meant two incompatible things

formatAmbiguousMatchCandidateLines needed three defensive guards and a 14-line comment whose own words were "Both guards below … must hold together, or this renders [object Object]", because details.candidates was string[] from buildAmbiguousMatchError and {id, name}[] from findBootedAppleSimulatorWithApp.

The device domain now owns its own key, devices. The renderer moves to src/utils/error-candidates.ts, which declares both shapes (ElementMatchCandidateDetails, DeviceCandidateDetails) next to the rendering, and both producers construct through those types. What is left is one narrowing per key, at the JSON wire boundary, with no cross-shape reasoning and no [object Object] hazard to explain.

This is the one intended visible change. The device shape previously rendered nothing at all on both the CLI and MCP text paths. It now renders, udid-first, matching what its own hint asks for:

Error (AMBIGUOUS_MATCH): Multiple booted iOS simulators have com.example.app installed
Hint: Pass --udid to select the intended simulator explicitly.
Devices:
  SIM-001  iPhone 17 Pro
  SIM-002  iPhone 17

The find handler's AMBIGUOUS_MATCH details are byte-identical, key order included (locator, query, matches, candidates). Skew-safe by construction: a daemon predating the devices key sends device objects under candidates, which the string reader filters to empty — exactly today's "render nothing", never [object Object]. A regression test pins that. MCP declares no schema for error details (command-output-schemas.ts only schemas the maestro response fields, unchanged), so no schema surface moved.

Validation

Behavior preservation for the Finding A unification — proven red first. The suppression rule is a 2×2: {runner-payload, runtime} × {fallback executed, not executed}. Three cells already existed in maestro-fallback.contract.test.ts; the fourth (runtime / not executed) was only covered in a distant file, so it is added here and all four now sit together.

Against a deliberately wrong predicate, each direction reddens exactly one cell per branch — both branches, both directions:

  • suppressesResolutionDisclosure → false (never suppress): 2 failed | 3 passed. Red on responseConstruction: fallback tap response … (runner-payload) and Maestro fill of a non-hittable input … (runtime), both with + { kind: 'not-observed', source: 'direct-ios' } where undefined was expected.
  • suppressesResolutionDisclosure → true (always suppress): 2 failed | 3 passed. Red on allowed-but-not-taken discloses direct-ios not-observed (runner-payload) and runtime fill the coordinate fallback did not execute keeps its resolution disclosure (runtime).
  • Correct predicate: 5 passed.

The route-parity guard was likewise proven red: dropping 'xctest-application-fallback' from TEXT_ENTRY_ROUTES fails with + "xctest-application-fallback" against the Swift sources.

Gates. pnpm typecheck, pnpm lint, pnpm format:check, pnpm check:layering (1020 files, R2–R11 green after pinning the three new contracts/interaction façade symbols), pnpm check:production-exports (no issues), pnpm test:unit (exit 0 — 611 files, 5390 tests), test/integration/interaction-contract/ (8 files, 64 tests), src/__tests__/contracts/ (22 tests). pnpm check:affected --run passed clean on this commit: 403 files, 3801 tests, "all runnable checks passed".

Residual, environmental. Later check:affected runs on this host flaked on provider-scenarios/android-lifecycle.test.ts, android-recording.test.ts, and doctor.test.ts — always a 15s Test timed out, never an assertion, with a different subset each run and all passing in isolation. Reproduced identically on a detached plain origin/main (same files, same tests, same timeout shape), so it is host load under coverage instrumentation, not this diff. AGENTS.md contention policy. Verify the CI Integration Tests job on this PR head.

No device verification: this is a type-level refactor with no device-facing behavior change. No docs or skills updated — no command surface, flag, help text, or output contract moved except Finding C's device block, which is error rendering and not documented anywhere.

19 files, +458/−162. Scope stayed inside the interaction dispatch/response family plus the shared error renderer; no expansion beyond the three findings.

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.97 MB 1.97 MB +617 B
JS gzip 631.4 kB 631.7 kB +226 B
npm tarball 759.5 kB 760.1 kB +604 B
npm unpacked 2.67 MB 2.67 MB +1.6 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 24.8 ms 25.7 ms +0.9 ms
CLI --help 59.1 ms 62.0 ms +2.9 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/viewport-dimension.js +312 B +70 B
dist/src/screenshot-geometry.js +133 B +56 B
dist/src/interaction.js +66 B +40 B
dist/src/session.js -7 B +5 B
dist/src/screenshot-result.js 0 B +2 B

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Reviewed exact head 9536876a3: no code or ADR blocker. Maestro fallback suppression remains usage-based across both response paths; runner type narrows at the Apple wire boundary with Swift-route parity; and the unreleased device-candidate shape now has a distinct devices key shared by CLI/MCP rendering.

Not ready yet:

  1. Android Smoke is red on the unrelated Active application interaction viewport is unavailable scroll canary; I requested a failed-job rerun.
  2. The intended visible Devices: change needs exact-head live evidence. Boot two iOS simulators, run open for a deliberately absent bundle, and capture the real CLI APP_NOT_INSTALLED output listing both UDIDs. Include MCP text evidence as well to support the cross-surface claim.

All other checks are green.

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Live evidence at exact head 9536876a3, from a CLI built from this branch (pnpm build in the branch worktree, invoked via bin/agent-device.mjs), against three booted simulators.

CLI

$ node bin/agent-device.mjs open com.example.definitely-not-installed --platform ios
Error (APP_NOT_INSTALLED): No booted iOS simulator has com.example.definitely-not-installed installed
Hint: Install the app on a booted simulator, or pass --udid to select the intended device explicitly.
Devices:
  9B8E796C-363C-4307-BEF7-BE91DF109672  ad-evidence-16pro
  C4578E05-BB63-4A3D-AB53-B011EFA2D981  bench-golden-bsky-latest
  6044A251-23C2-4584-B7DB-87A16B787757  iPhone 17 Pro

MCP — same build, mcp server over stdio, tools/callopen with platform: ios:

Error (APP_NOT_INSTALLED): No booted iOS simulator has com.example.definitely-not-installed installed
Hint: Install the app on a booted simulator, or pass --udid to select the intended device explicitly.
Devices:
  9B8E796C-363C-4307-BEF7-BE91DF109672  ad-evidence-16pro
  C4578E05-BB63-4A3D-AB53-B011EFA2D981  bench-golden-bsky-latest
  6044A251-23C2-4584-B7DB-87A16B787757  iPhone 17 Pro

Byte-identical across both surfaces, udid-first, which is the cross-surface claim. Each run used an isolated --state-dir so no pre-existing daemon could serve the response.

Note the run without --platform ios routes to Android and never reaches findBootedAppleSimulatorWithApp, so the iOS resolver has to be selected explicitly for this path to fire — worth knowing if you re-run it.

🤖 Addressed by Claude Code

@thymikee
thymikee marked this pull request as ready for review August 5, 2026 11:05
…ls real types

Three places smuggled structured data through untyped bags and re-read it
with runtime guards. Each gets an explicit typed boundary.

A. The resolution-suppression rule was encoded twice in
   interaction-touch-response.ts — a spread ternary in the runner-payload
   branch and an unconditional destructure used conditionally in the runtime
   branch, with the ADR 0012 rationale living on only one source variant.
   Both branches now read one `suppressesResolutionDisclosure(source)`
   predicate through one `applyResolutionDisclosurePolicy` helper, where the
   reason is stated once. The union field is renamed
   `maestroCoordinateFallbackDispatched` (the dispatch path that ran) and
   hoisted into a shared base. handleFillCommand's two-arm interactor.fill
   call collapses to one.

B. `Interactor.type` narrows from `Record<string, unknown> | void` to
   `TypeTextBackendResult | void`; the Apple runner boundary is the single
   place the wire payload becomes that type. `maestroFallbackDetails` returns
   a typed `{ used, extra }` instead of a bag both call sites re-read.

C. `details.candidates` meant two incompatible things. The device-domain
   resolvers now key their list `devices`, so the shared renderer drops its
   shape-disambiguation guards and the device list actually renders.
@thymikee
thymikee force-pushed the claude/type-interaction-boundaries branch from 9536876 to 50b7db4 Compare August 5, 2026 11:10
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Re-captured at the rebased exact head 50b7db4fb (the branch was rebased onto 9fc2663; the previous evidence cited 9536876a3 and is now stale). Same CLI built from this branch, same three booted simulators, both surfaces byte-identical:

Error (APP_NOT_INSTALLED): No booted iOS simulator has com.example.definitely-not-installed installed
Hint: Install the app on a booted simulator, or pass --udid to select the intended device explicitly.
Devices:
  9B8E796C-363C-4307-BEF7-BE91DF109672  ad-evidence-16pro
  C4578E05-BB63-4A3D-AB53-B011EFA2D981  bench-golden-bsky-latest
  6044A251-23C2-4584-B7DB-87A16B787757  iPhone 17 Pro

CLI via bin/agent-device.mjs, MCP via the mcp stdio server → tools/call open with platform: ios. Isolated --state-dir on each run.

🤖 Addressed by Claude Code

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 50b7db4fb: the rebase preserves the reviewed patch exactly (identical stable patch ID and file stats), all CI is green, and the new exact-head branch-built CLI plus MCP evidence over three booted simulators fully proves the visible Devices: rendering on both surfaces. The prior Android and stale-evidence blockers are resolved. No remaining code, evidence, CI, or branch blockers.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 5, 2026
@thymikee
thymikee merged commit ee473b6 into main Aug 5, 2026
31 checks passed
@thymikee
thymikee deleted the claude/type-interaction-boundaries branch August 5, 2026 12:32
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-05 12:32 UTC

thymikee added a commit that referenced this pull request Aug 5, 2026
Two review findings, plus a third the gate caught on itself.

P1 — the three `DEFAULT_DRAG_*` constants join the existing public-façade
suppression, alongside `COORDINATE_GESTURE_KINDS` and
`normalizePublicGesture` which the same conversion surfaced. All five are
#1567's drag vocabulary, made individually visible to `--production`
analysis for the first time because a bare star used to hide them from
that exact check. Kept rather than narrowed, for the reason the existing
entry already states: the façade's surface stays byte-identical to what
the retired pin table asserted, and narrowing is a follow-up with its own
review.

P2 — the exhaustiveness gate skipped any source carrying a bare
`export *`, which dropped that module's DIRECT exports from the check too.
`gesture-plan.ts` stars `gesture-plan-types.ts`, so removing
`buildDragGesturePlan` from the façade narrowed the public surface and
still passed. `readDirectNamedExports` now reads exactly the names a module
declares or re-exports BY NAME and ignores the star, so direct exports are
checked while the starred set stays covered by the façade's own direct
re-export of that module.

Red evidence: removing `buildDragGesturePlan` from facades/interaction.ts
now fails naming file, source and symbol; 13 pass / 0 fail restored.

Third, and the reason the gate is worth having: rebasing onto main after
#1612 merged silently dropped `TEXT_ENTRY_ROUTES`, `TextEntryRoute` and
`TypeTextBackendResult` from the interaction façade — the same narrowing
class as the #1567 one review caught by hand, one merge later. The gate
failed on it before CI did. Restored.
thymikee added a commit that referenced this pull request Aug 5, 2026
…n table (#1614)

* refactor(contracts): name façade exports explicitly and retire the pin table

Thirteen of the fourteen `@agent-device/contracts` façades were bare
`export *` barrels. `facades/snapshot.ts`, added by #1582, was the one
exception — explicit named re-exports — and that is now the rule.

Everything #1574 built to cope with `export *` goes with them:

  scripts/layering/facade-symbols.ts          -980   (816 pinned names)
  scripts/layering/facade-exports.ts          -192   (readFacadeExports)
  scripts/layering/facade-exports.test.ts     -234   (star semantics)
  scripts/layering/package-boundaries.test.ts  -55

`readFacadeExports` re-implemented ESM `GetExportedNames`/`ResolveExport`
— star-chain resolution, ambiguity rejection, diamond binding identity,
cycle guards, spec-accurate `default` filtering at the star rather than
the source. All of it existed to enumerate what `export *` hides. 523 of
the 816 pinned names belonged to contracts, i.e. to those thirteen files.
Once a façade names its exports, the façade file IS the pin, and it is
visible in the diff of the file that widened rather than in a separate
table a reviewer has to cross-check.

`readNamedExports` (20 lines) stays and is enough: it already throws on
bare `export *` and on `export default`. The pin is replaced by one
structural gate — no façade may contain a bare star — which reuses that
rejection rather than adding a regex.

Surface equivalence verified independently, not asserted: main's own
`readFacadeExports` run over the new façades, compared against main's own
`FACADE_SYMBOLS` table — 31 subpaths, 0 added, 0 removed.

Red evidence for the new gate: planting `export * from '../request-progress.ts'`
back into facades/progress.ts fails it with the file named and the reason
quoted; 12 pass / 0 fail once reverted.

Not included: the `lowerAndroidTouchPlan` tuple-assertion drive-by. It
needs `sampleGestureOffsets` to carry a min-arity tuple through `.map()`,
which TypeScript will not infer without a typed helper — a real change to
the gesture-plan contract rather than a drive-by, so it stays out.

* test(layering): assert façades stay exhaustive over their sources

Review on #1614 caught this conversion silently narrowing the public
surface. The explicit lists were generated against the surface at fork
time; #1567 landed 13 exports meanwhile — `DragOptions`, the drag-gesture
vocabulary (`COORDINATE_GESTURE_KINDS`, `CoordinateGesturePayload`, the
three `DEFAULT_DRAG_*` constants, `DragGestureInput`, `DragGesturePayload`,
`GestureCommandInput`, `buildDragGesturePlan`,
`dragGesturePayloadFromPositionals`, `normalizeGestureCommandInput`) and
`MultiTargetAnnotationV1`. The `export *` barrels had been forwarding all
13 automatically; the rebase dropped every one, and only a human diff
caught it.

The star-rejection gate could not: it only proves a façade does not WIDEN
invisibly. Narrowing is the failure an explicit list newly makes possible,
because `export *` could not narrow by construction. So the property the
stars gave for free is now asserted directly — every name a re-exported
source declares must appear in the façade.

Scoped to `packages/*/src/facades/`, the barrels this PR converted. A
hand-curated package `index.ts` is a different thing: `ad-replay`
deliberately publishes two values out of a much larger `internal/`, and
forcing exhaustiveness there would widen a surface its owner narrowed on
purpose (#1555). A source that itself carries a bare `export *` is skipped
— unknowable from that file alone, and reachable because the façade
re-exports the starred module directly too, which IS checked.

Red evidence: dropping `MultiTargetAnnotationV1` from facades/replay.ts —
one of the 13 the old gate was blind to — fails with the file, the source
and the symbol named. 13 pass / 0 fail once restored.

* fix(layering): close the exhaustiveness gate's starred-source hole

Two review findings, plus a third the gate caught on itself.

P1 — the three `DEFAULT_DRAG_*` constants join the existing public-façade
suppression, alongside `COORDINATE_GESTURE_KINDS` and
`normalizePublicGesture` which the same conversion surfaced. All five are
#1567's drag vocabulary, made individually visible to `--production`
analysis for the first time because a bare star used to hide them from
that exact check. Kept rather than narrowed, for the reason the existing
entry already states: the façade's surface stays byte-identical to what
the retired pin table asserted, and narrowing is a follow-up with its own
review.

P2 — the exhaustiveness gate skipped any source carrying a bare
`export *`, which dropped that module's DIRECT exports from the check too.
`gesture-plan.ts` stars `gesture-plan-types.ts`, so removing
`buildDragGesturePlan` from the façade narrowed the public surface and
still passed. `readDirectNamedExports` now reads exactly the names a module
declares or re-exports BY NAME and ignores the star, so direct exports are
checked while the starred set stays covered by the façade's own direct
re-export of that module.

Red evidence: removing `buildDragGesturePlan` from facades/interaction.ts
now fails naming file, source and symbol; 13 pass / 0 fail restored.

Third, and the reason the gate is worth having: rebasing onto main after
#1612 merged silently dropped `TEXT_ENTRY_ROUTES`, `TextEntryRoute` and
`TypeTextBackendResult` from the interaction façade — the same narrowing
class as the #1567 one review caught by hand, one merge later. The gate
failed on it before CI did. Restored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant