Skip to content

refactor(ios): share one private-XCTest event bridge between gesture and text synthesis - #1608

Merged
thymikee merged 1 commit into
mainfrom
claude/runner-xctest-bridge
Aug 5, 2026
Merged

refactor(ios): share one private-XCTest event bridge between gesture and text synthesis#1608
thymikee merged 1 commit into
mainfrom
claude/runner-xctest-bridge

Conversation

@thymikee

@thymikee thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member

Summary

RunnerSynthesizedGesture.m and RunnerSynthesizedTextEntry.m (added in #1588) each independently reflected into the same private XCTest event-synthesis API (XCSynthesizedEventRecord / XCPointerEventPath). A selector rename in that private surface had to be found and fixed in two files with two different failure shapes. This PR extracts the duplicated resolution into a shared RunnerXCTestEventBridge.h / .m.

Unified into the shared bridge:

  • The RunnerXCTestEventBridge struct holding the common core: recordClass/pathClass, addPointerEventPath:, setTargetProcessID:, synthesizeWithError:, and processID.
  • RunnerResolveXCTestEventBridge, which resolves and validates that core against the live XCTest runtime.
  • RunnerRequireClass / RunnerRequireSelector / RunnerRequireApplicationSelector — now take a surface parameter ("event" or "text") so each caller's existing message wording ("private XCTest event synthesis unavailable: ..." vs "private XCTest text synthesis unavailable: ...") is preserved byte-for-byte instead of being forced to one wording.
  • The shared RunnerMsgSend* objc_msgSend typedefs both callers use (RunnerMsgSendInteger, RunnerMsgSendSetInteger, RunnerMsgSendAddPath, RunnerMsgSendSynthesize).
  • RunnerFormatXCTestException, the @catch (NSException *)"name: reason" formatter that was byte-identical in both files.

Deliberately kept per-caller (not force-unified):

  • The record-init overload: gesture uses the 2-arg initWithName:interfaceOrientation: (plus interfaceOrientation and the touch-path selectors initForTouchAtPoint:offset: / moveToPoint:atOffset: / liftUpAtOffset:); text entry uses the 1-arg initWithName: (plus the text-input selectors initForTextInput / typeText:atOffset:typingSpeed:shouldRedact: / typeKey:modifiers:atOffset:, the last only required in replace mode). These are genuinely different selectors, not a naming coincidence, so each file resolves its own extension of the shared core (RunnerGestureEventBridge / RunnerTextEventBridge, each embedding RunnerXCTestEventBridge core).
  • The external result shape: RunnerSynthesizedGesture still returns a nullable NSString * message; RunnerSynthesizedTextEntry still returns a RunnerSynthesizedTextEntryResult with a status enum. Swift-visible signatures are unchanged, so AgentDeviceRunnerUITests-Bridging-Header.h did not need a change.
  • Unifying those two error-reporting shapes (nullable string vs. status enum) is a real follow-up worth doing but out of scope here.

Pure deduplication — no behavior change. Net: +318/-202 across the 4 touched files (2 new, 2 edited).

Validation

  • pnpm typecheck, pnpm lint, pnpm format:check: all clean (no TS files touched).
  • pnpm exec vitest run src/__tests__/apple-runner-package-source.test.ts: 5/5 passing — the new .h/.m files are picked up automatically by the packaging script's directory walk (no manifest to update).
  • pnpm test:unit: 610 files / 5382 tests passing.
  • Runner compile evidence (this is a private-API objc_msgSend refactor, so this is the load-bearing gate): built both shared targets locally with a real Xcode toolchain (Xcode 26.2) since this changes code shared between iOS and macOS per docs/agents/testing.md:
    • pnpm build:xcuitest:macos -> ** TEST BUILD SUCCEEDED **, clang compiled RunnerXCTestEventBridge.m, RunnerSynthesizedGesture.m, and RunnerSynthesizedTextEntry.m with zero errors/warnings.
    • pnpm build:xcuitest:ios (iphonesimulator destination) -> ** TEST BUILD SUCCEEDED **, same three files compiled cleanly.
  • pnpm check:affected --run: selects only the GitHub-authoritative swift-runner check (already covered by the local builds above); reports "all runnable checks passed."
  • The project uses Xcode 16's PBXFileSystemSynchronizedRootGroup for this test target, so the new files needed no project.pbxproj edit — they were picked up automatically by both the local builds and by check:affected's file-based classification.

Follow-up (optional, out of scope)

Unify RunnerSynthesizedGesture's nullable-message-string error shape with RunnerSynthesizedTextEntry's status-enum shape, if a future caller needs to distinguish "unavailable" from "failed" for gestures the way text entry already can.

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.97 MB 1.97 MB 0 B
JS gzip 631.7 kB 631.7 kB 0 B
npm tarball 760.3 kB 761.6 kB +1.3 kB
npm unpacked 2.67 MB 2.67 MB +2.7 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 23.9 ms 23.6 ms -0.3 ms
CLI --help 56.4 ms 56.9 ms +0.4 ms

Top changed chunks: no changes in the largest emitted chunks.

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Reviewed exact head ba6be1775: no source or architecture blocker found. The extraction preserves the duplicated private-XCTest core, caller-specific selectors/result shapes, and existing error wording; CI is fully green and the new translation unit compiles, links, and packages on iOS/macOS.

One readiness requirement remains: current executed tests do not invoke the non-stubbed bridge. Before undrafting/labeling, please add exact-head simulator evidence for one synthesized coordinate tap and one private synthesized text-entry operation, followed by another command confirming runner usability.

@thymikee
thymikee force-pushed the claude/runner-xctest-bridge branch from ba6be17 to 8b7febb Compare August 5, 2026 11:10
…and text synthesis

RunnerSynthesizedGesture.m and RunnerSynthesizedTextEntry.m each independently
reflected into XCSynthesizedEventRecord/XCPointerEventPath. Extract the
common resolution (classes, addPointerEventPath:/setTargetProcessID:/
synthesizeWithError:/processID, the RunnerRequireClass/Selector/
ApplicationSelector checks, the shared objc_msgSend typedefs, and the
"name: reason" exception formatter) into RunnerXCTestEventBridge.h/.m.
Each caller resolves its own extras on top of the shared core: gesture
resolves the 2-arg initWithName:interfaceOrientation: plus touch-path
selectors, text entry resolves the 1-arg initWithName: plus text-input
selectors.
@thymikee
thymikee force-pushed the claude/runner-xctest-bridge branch from 8b7febb to 0dfe07a Compare August 5, 2026 12:40
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

1. Rebased onto current main (past #1616)

#1616 landed a real consolidation in RunnerSynthesizedGesture.m (RunnerCreateEventRecord / RunnerSynthesizeEventRecord, RunnerTrySynthesizeTap(id application, CGPoint point)) that I re-derived my extraction on top of instead of reverting.

Resolution:

  • RunnerCreateEventRecord / RunnerSynthesizeEventRecord stay in RunnerSynthesizedGesture.m, retyped to take RunnerGestureEventBridge * (the gesture-local extension of the shared core) instead of the old flat RunnerXCTestEventBridge.
  • RunnerTrySynthesizeTap(id application, CGPoint point) kept as docs: clarify iOS drag synthesis profiles #1616 shaped it; its body now resolves the bridge and formats exceptions through the shared header.
  • Judgement call on RunnerCreateEventRecord/RunnerSynthesizeEventRecord generic reuse (asked for explicitly): left both gesture-local, not moved to the shared file. RunnerCreateEventRecord hardcodes the 2-arg initWithName:interfaceOrientation: overload and unconditionally reads interfaceOrientation — text entry uses the 1-arg initWithName: and never reads orientation, and unlike gesture it sometimes creates two records in one operation (selection + main) against one resolved bridge. RunnerSynthesizeEventRecord also bakes in one fixed error string ("private XCTest event synthesis failed: %@"), whereas text entry needs two different wordings at two call sites ("text synthesis failed" vs "text selection failed"). Parameterizing both for reuse across files would be exactly the kind of false unification the task warned against, for a 2-line saving in one file. Text entry's own two synthesize-then-format-error call sites staying near-identical within that one file is a smaller, separate, in-file pattern — not touched here.

Rebase command and result:

$ git rebase origin/main
Rebasing (1/1)Auto-merging apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerSynthesizedGesture.m
CONFLICT (content): Merge conflict in apple/runner/.../RunnerSynthesizedGesture.m
error: could not apply 8b7febb61... refactor(ios): share one private-XCTest event bridge between gesture and text synthesis

Only that one file conflicted (RunnerSynthesizedTextEntry.m and the two new RunnerXCTestEventBridge.* files applied cleanly). After hand-resolving:

$ git rebase --continue
[detached HEAD 0dfe07ab0] refactor(ios): share one private-XCTest event bridge between gesture and text synthesis
 4 files changed, 312 insertions(+), 195 deletions(-)
Successfully rebased and updated refs/heads/claude/runner-xctest-bridge.

New head: 0dfe07ab0b2ed122e8c8c7c2433f59e1b7ec5c16.

Gates re-run at the rebased head

$ pnpm typecheck
$ tsc -b packages/xml packages/kernel packages/contracts packages/ad-script packages/selectors packages/ad-replay packages/maestro packages/replay-test packages/provider-webdriver packages/provider-limrun && tsc -p tsconfig.json && tsc -p examples/sdk/tsconfig.json
(clean, no output)

$ pnpm lint
$ oxlint . --deny-warnings
(clean, no output)

$ pnpm format:check
$ node ./node_modules/oxfmt/bin/oxfmt --check
Checking formatting...
All matched files use the correct format.
Finished in 489ms on 2103 files using 12 threads.

$ pnpm exec vitest run src/__tests__/apple-runner-package-source.test.ts
 ✓ |unit-core| src/__tests__/apple-runner-package-source.test.ts (5 tests) 173ms
 Test Files  1 passed (1)
      Tests  5 passed (5)

Compile evidence (real Xcode toolchain, Xcode 26.2)

$ pnpm build:xcuitest:macos
... clang compiles RunnerXCTestEventBridge.m, RunnerSynthesizedGesture.m, RunnerSynthesizedTextEntry.m ...
** TEST BUILD SUCCEEDED **

$ pnpm build:xcuitest:ios
... clang compiles the same three files for the iphonesimulator SDK ...
** TEST BUILD SUCCEEDED **
$ pnpm check:affected --run
Selected 1 check(s):
  - swift-runner: Swift runner build (GitHub-authoritative; not run locally)
      · apple/.../RunnerSynthesizedGesture.m [own:swift]
      · apple/.../RunnerSynthesizedTextEntry.m [own:swift]
      · apple/.../RunnerXCTestEventBridge.h [own:swift]
      · apple/.../RunnerXCTestEventBridge.m [own:swift]
[skip] swift-runner — GitHub-authoritative (jobs: Swift Runner Unit Compile, iOS / Smoke Tests, macOS / Smoke Tests)
check:affected: all runnable checks passed.

Force-pushed the rebased branch:

$ git push --force-with-lease origin claude/runner-xctest-bridge
 + 8b7febb61...0dfe07ab0 claude/runner-xctest-bridge -> claude/runner-xctest-bridge (forced update)

2. Simulator evidence at head 0dfe07ab0

Built the CLI (pnpm build) and the iOS runner (pnpm build:xcuitest:ios) from this worktree, then drove them with node bin/agent-device.mjs (not a global install) against an isolated --state-dir.

Device note: the iPhone 17 Pro UDID I was handed (6044A251-...) was claimed by a concurrent sibling daemon (PID 96079, AGENT_DEVICE_STATE_DIR=/tmp/ad-ev-1615 — a different worktree's live agent-device daemon) between my open and my first runner command, and shortly after that no simulator was booted at all — real contention from a parallel agent, not something this change caused. I closed that session immediately and instead booted a throwaway dedicated simulator (xcrun simctl create/boot, iPhone 17 Pro / iOS 26.2, UDID 9991F66D-3283-429A-9779-89DF05678142) so the run is uncontended, then deleted it afterward (xcrun simctl shutdown + delete, confirmed gone).

$ git rev-parse HEAD
0dfe07ab0b2ed122e8c8c7c2433f59e1b7ec5c16

$ node bin/agent-device.mjs open Safari --platform ios --udid 9991F66D-3283-429A-9779-89DF05678142 --state-dir /tmp/ad-ev-1608 --session ev1608 --json
{
  "success": true,
  "data": {
    "session": "ev1608",
    "runnerLogPath": "/tmp/ad-ev-1608/sessions/ev1608/runner.log",
    "appName": "Safari",
    "appBundleId": "com.apple.mobilesafari",
    "device": "ad-ev-1608",
    "device_udid": "9991F66D-3283-429A-9779-89DF05678142",
    "message": "Opened: Safari"
  }
}

Coordinate tap through synthesized gesture synthesis

$ node bin/agent-device.mjs snapshot -i --session ev1608 --state-dir /tmp/ad-ev-1608
...
@e23 [text-field] "Address" [editable]
...

$ node bin/agent-device.mjs press @e23 --session ev1608 --state-dir /tmp/ad-ev-1608 --json
{
  "success": true,
  "data": {
    "x": 203, "y": 816,
    "message": "Tapped @e23 (203, 816)",
    "ref": "e23",
    "targetHittable": false
  }
}

$ grep AGENT_DEVICE_RUNNER_SYNTHESIZED_GESTURE_POLICY /tmp/ad-ev-1608/sessions/ev1608/runner.log
AgentDeviceRunnerUITests-Runner[96787:96055643] AGENT_DEVICE_RUNNER_SYNTHESIZED_GESTURE_POLICY kind=coordinateTap axHealth=healthy frameSource=screenshot keyboardPolicy=never fallbackPolicy=xctestCoordinateAllowed fallbackAllowed=true fallbackAttempted=false

fallbackAttempted=false means the tap went through RunnerSynthesizedGesture.synthesizeTap (the code this PR touched, via the rebased RunnerTrySynthesizeTap/shared bridge) and succeeded without falling back to the plain XCTest coordinate tap.

Text entry through the private synthesized route

$ node bin/agent-device.mjs type "agent-device-pr1608-evidence" --session ev1608 --state-dir /tmp/ad-ev-1608 --json
{
  "success": true,
  "data": {
    "textEntryRoute": "synthesized-first-responder",
    "text": "agent-device-pr1608-evidence",
    "message": "Typed 28 chars"
  }
}

$ grep AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE /tmp/ad-ev-1608/sessions/ev1608/runner.log
AgentDeviceRunnerUITests-Runner[96787:96055643] AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder
AgentDeviceRunnerUITests-Runner[96787:96055643] AGENT_DEVICE_RUNNER_TEXT_ENTRY_ROUTE route=synthesized-first-responder

route=synthesized-first-responder is the RunnerSynthesizedTextEntry.synthesizeText path this PR refactored — not xctest-element and not xctest-application-fallback. A screenshot right after confirms the text actually landed in Safari's address/search field (visible as agent-device-pr1608-evidence in the Google-suggestions row).

Further command confirming runner usability

$ node bin/agent-device.mjs screenshot /tmp/ad-ev-1608/evidence-screenshot.png --session ev1608 --state-dir /tmp/ad-ev-1608 --json
{
  "success": true,
  "data": { "path": "/tmp/ad-ev-1608/evidence-screenshot.png", "width": 402, "height": 874 }
}

Session and daemon closed cleanly afterward:

$ node bin/agent-device.mjs close --session ev1608 --state-dir /tmp/ad-ev-1608 --json
{ "success": true, "data": { "session": "ev1608", "message": "Closed: ev1608" } }

$ node bin/agent-device.mjs daemon stop --state-dir /tmp/ad-ev-1608 --json
{ "success": true, "data": { "stopped": true, "mode": "graceful" } }

$ xcrun simctl shutdown 9991F66D-3283-429A-9779-89DF05678142 && xcrun simctl delete 9991F66D-3283-429A-9779-89DF05678142
$ xcrun simctl list devices | grep ad-ev-1608
(no output — device removed)

🤖 Addressed by Claude Code

@thymikee
thymikee marked this pull request as ready for review August 5, 2026 13:10
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Undrafting at 0dfe07ab0. Verified the rebase resolution independently of the worker's report:

  • Reflected API surface is byte-identical to main. The union of NSClassFromString/NSSelectorFromString across the three files on this branch is exactly the set across the two files on main — 15 entries, none added, dropped or renamed. For a private-API extraction that is the check that matters: the refactor moves where the reflection lives, not what it reflects.
  • docs: clarify iOS drag synthesis profiles #1616 survived the resolution. RunnerCreateEventRecord / RunnerSynthesizeEventRecord are still there (10 references) and RunnerTrySynthesizeTap(id, CGPoint) keeps its new signature. The extraction was re-derived on top of that consolidation rather than reverting it to make the patch apply.
  • Compiled, not assumed. pnpm build:xcuitest:ios and :macos both printed ** TEST BUILD SUCCEEDED ** at this head.
  • Simulator evidence at this exact head is in the comment above: synthesized tap with fallbackAttempted=false, text entry on route=synthesized-first-responder (the private path, not a fallback), and a follow-up command proving runner usability. Throwaway simulator created for it was deleted afterwards.

🤖 Addressed by Claude Code

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 0dfe07ab0b2ed122e8c8c7c2433f59e1b7ec5c16 after the rebase: no code-review blockers found.

The conflict resolution preserves #1616’s lifecycle helpers and tap signature, while the reflected private-XCTest surface remains exactly the same 15 symbols as main. Gesture and text-entry caller-specific selectors, validation, and error behavior remain intact. Exact-head simulator evidence exercises the synthesized tap path, private synthesized text entry, and a successful follow-up command, with the isolated state/device cleaned up afterward. All current CI checks are green.

This head is ready for human merge review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 5, 2026
@thymikee
thymikee merged commit 959d858 into main Aug 5, 2026
28 checks passed
@thymikee
thymikee deleted the claude/runner-xctest-bridge branch August 5, 2026 13:57
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-05 13:57 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant