Skip to content

test(ios): fix nightly fixture selector drift - #1492

Merged
thymikee merged 2 commits into
mainfrom
agent/fix-nightly-ios-selector-drift
Jul 30, 2026
Merged

test(ios): fix nightly fixture selector drift#1492
thymikee merged 2 commits into
mainfrom
agent/fix-nightly-ios-selector-drift

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

Fix the full fixture-backed iOS nightly assertion to target automation-press, matching the fixture app and the healthy post-app-switcher snapshot captured by the failed run.

Add a host-side coverage guard that verifies literal identifiers used by live iOS scenarios exist as fixture-screen testID values. Future selector drift now fails the regular Node integration lane instead of waiting for nightly device CI.

Root cause: the newly added full-tier scenario used the nonexistent identifier automation-press-canary; agent-device correctly returned automation-press with the app inactive and all buttons non-hittable. No runtime behavior, retries, timeouts, or workflows changed.

Scope stayed within the iOS E2E test family: 2 files touched.

Validation

  • Reproduced the exact failure with the new guard before the fix: automation-press-canary was the sole missing fixture identifier.
  • pnpm check:affected --run passed formatting, lint, typecheck, fallow, related-test selection, and the Node integration lane (28 passed, 7 skipped).
  • The daemon-clean integration test passed in isolation after one contention-shaped failure during the first aggregate attempt.

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.88 MB 1.88 MB -4.9 kB
JS gzip 605.4 kB 604.0 kB -1.5 kB
npm tarball 724.0 kB 722.5 kB -1.5 kB
npm unpacked 2.53 MB 2.53 MB -4.9 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.1 ms 27.3 ms -0.8 ms
CLI --help 56.6 ms 57.6 ms +1.0 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/session.js -3.5 kB -984 B
dist/src/internal/daemon.js -1.3 kB -457 B
dist/src/selector-runtime.js +19 B +4 B

@thymikee

Copy link
Copy Markdown
Member Author

Exact head d96938a is not ready. The selector fix itself matches the healthy fixture (automation-press), but the new drift guard does not enforce its stated contract: /id=\\"([^"$]+)\\"/g requires literal backslashes and therefore misses ordinary source selectors such as \x27id="automation-last-nonactive"\x27. It currently catches only the identifier === \x27automation-press\x27 form, and it omits live-runner.ts, where live form selectors are owned. All PR checks are green, but the PR iOS job runs the smoke tier; the changed lifecycle assertion runs only in the nightly full tier. Correct the guard so a direct-selector typo makes it fail, include all owned live scenario sources, and provide a fresh exact-head full-tier iOS run before readiness.

@thymikee

Copy link
Copy Markdown
Member Author

Clean re-review at 7253f36. The guard now matches ordinary single- and double-quoted literal id=… selectors and snapshot identifier === … checks, with a focused counterfactual proving typos fail. It scans every live-*.ts iOS E2E source, closing the prior ownership gap. All exact-head PR checks are green, and manually dispatched Replay Nightly run 30530012666 succeeded on this exact SHA, including the full fixture-backed iOS simulator tier that exercises the repaired lifecycle assertion. No remaining findings; ready for human merge.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 30, 2026
@thymikee
thymikee merged commit 117cf4f into main Jul 30, 2026
31 checks passed
@thymikee
thymikee deleted the agent/fix-nightly-ios-selector-drift branch July 30, 2026 10:14
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-30 10:15 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant