Skip to content

test(web): read crosscheck pages with the clock paused - #1555

Merged
okwasniewski merged 1 commit into
mainfrom
oskar/crosscheck-stale-dom
Nov 5, 2026
Merged

okwasniewski merged 1 commit into
mainfrom
oskar/crosscheck-stale-dom

Conversation

@okwasniewski

@okwasniewski okwasniewski commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

The web crosscheck sometimes failed on benchmark: stale-dom with getByRole button "Decoy" / "Tap me": ours match, theirs no match (seen on #1552, gone on rerun). It was a race in the harness, not a regression in the engine.

  • stale-dom remounts every row on a 1s setInterval. The crosscheck tags the engine's elements, reads Chrome's AX tree, and then asks Playwright, all on the live page. When a tick landed between those reads, Playwright was asked about tagged buttons that had already been detached.
  • Reproduced at random tick phase under 8x CPU throttle: 8/40 runs on main, the same at HEAD~4 (before fix(e2e): let setInputFiles([]) clear a file input #1546-fix(web): screenshot past a frame whose navigation is held #1551). None of those merges caused it.
  • Fix: openCheckPage (packages/web/tests/crosscheck/crosscheck.ts) installs Playwright's clock paused before any document loads. Timers never fire, so the engine and both oracles read the same document. React and Next.js still hydrate (their scheduler uses MessageChannel), and expected.txt is unchanged.
  • No retries or margins. Pausing the clock mid-page can't target "now" (pauseAt throws on a past time), and CDP virtual time stalled Playwright calls, so the clock is paused from the start.
  • Coverage cost: the reviewer diffed all 42 scenarios. Only overlay-trap changed, losing a close button that a 500ms timeout reveals and that was only checked when the timeout beat networkidle.
  • New fixture list rebuilt every millisecond guards the race. AGENTS.md notes that content revealed by a timer is never read.

Verified

Ran it locally: yes

  • stale-dom through crossCheck 40 times, random phase, 8x CPU throttle:
main this branch
8/40 runs report the two getByRole misses 0/40, same 9 nodes compared
  • New fixture with the clock install removed: fails with chrome unknown / missing on the rebuilt buttons. With it: passes.
  • Full crosscheck against a local benchmark (CROSSCHECK_BENCHMARK_URL): 19/19 passed, expected.txt untouched.

View guided diff Turn on auto-fix

@github-actions github-actions Bot added the size:XS Up to 9 changed lines, not counting tests, fixtures, or the lockfile label Nov 4, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

View guided diff | Turn on auto-fix | Re-trigger cubic

@okwasniewski

okwasniewski commented Nov 4, 2026 •

Copy link
Copy Markdown
Member Author

[claude-opus-5-5] responding on behalf of okwasniewski

For the human: the web benchmark job, which runs the crosscheck step, was skipped here. .github/filters.yml keeps packages/*/tests/** from triggering benchmarks, so CI never ran this harness change against the benchmark. I ran it locally: the full crosscheck against next start of web-benchmark passed 19/19 with expected.txt unchanged. Main's next benchmark run after merge will be the first CI run. If harness changes should trigger web, that's a filters.yml decision.

stale-dom rebuilds its list every second; a rebuild between the engine read and
the oracle reads made getByRole miss the tagged, detached buttons.
@okwasniewski
okwasniewski force-pushed the oskar/crosscheck-stale-dom branch from 95fe309 to 9be4fdf Compare November 5, 2026 08:51
@okwasniewski
okwasniewski merged commit 76a5280 into main Nov 5, 2026
29 checks passed
@okwasniewski
okwasniewski deleted the oskar/crosscheck-stale-dom branch November 5, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:XS Up to 9 changed lines, not counting tests, fixtures, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant