Repository navigation
feat(e2e)!: inputValue and isChecked throw on the wrong element, uncheck clicks ARIA radios - #1552
Conversation
There was a problem hiding this comment.
All reported issues were addressed across 15 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
🟢 e2e web agent: 33 passed34 agent steps · 26 replayed from cache · 20 model calls · 87.5k tokens (94% cached) · $0.0034 All 33 tests in 5 files
Tested by e2e 0.22.0 · 2m 27s · web · run artifacts |
🟢 e2e ios agent: 21 passed, 1 skipped25 agent steps · 12 replayed from cache · 56 model calls · 350.2k tokens (82% cached) · $0.0237 All 22 tests in 1 file
Tested by e2e 0.22.0 · 10m 18s · ios-simulator · run artifacts |
e9d9a93 to
efcbbf2
Compare
🟢 e2e web: 91 passedAll 91 tests in 41 files
Tested by e2e 0.22.0 · 1m 13s · web · run artifacts |
🟢 e2e android agent: 20 passed24 agent steps · 15 replayed from cache · 37 model calls · 202.1k tokens (82% cached) · $0.0139 All 20 tests in 1 file
Tested by e2e 0.22.0 · 6m 46s · android-emulator · run artifacts |
🟢 e2e ios: 55 passed, 1 skippedAll 56 tests in 7 files
Tested by e2e 0.22.0 · 6m 2s · ios-simulator · run artifacts |
🟢 e2e android: 1 flaky, 56 passed, 1 skipped
|
| Test | Time | |
|---|---|---|
| tests/controls.e2e.ts · 1 flaky, 23 passed, 1 skipped | 2m 48s | |
| control inventory › a label query resolves a labeled text field (1 failed attempt first) | 7.0s | |
| 🟢 | control inventory › a labeled field answers name, display value, value reads, fill, and clear | 13.1s |
| 🟢 | control inventory › a placeholder query and attribute reads find the field | 6.7s |
| 🟢 | control inventory › a secure field takes a fill and denies value reads | 7.4s |
| 🟢 | control inventory › press sends a character and Enter through the soft keyboard | 8.6s |
| 🟢 | control inventory › a field autofocuses on mount and focus() moves the focus | 9.7s |
| 🟢 | control inventory › toBeFocused follows the focus | 9.4s |
| 🟢 | control inventory › a switch checks, unchecks, and reports its state | 8.7s |
| 🟢 | control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap | 7.1s |
| 🟢 | control inventory › checkbox and radio roles check, uncheck, and report their state | 11.0s |
| 🟢 | control inventory › a tab role answers getByRole | 4.3s |
| 🟢 | control inventory › a radio group reports the checked option | 8.4s |
| 🟢 | control inventory › buttons report disabled, enabled, and a state that arrives later | 7.4s |
| 🟢 | control inventory › a display: none element is absent rather than hidden, its visible twin attached | 6.7s |
| 🟢 | control inventory › a header answers its accessible name | 4.8s |
| 🟢 | control inventory › a tab row answers count, all, texts, filters, positions, and switching | 6.8s |
| 🟢 | control inventory › a list answers counts, list-form text, reads, and boxes in order | 6.1s |
| ⏭️ | control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…) | |
| 🟢 | control inventory › a long press lands as one | 8.0s |
| 🟢 | control inventory › a double tap lands inside the 300 ms window | 7.2s |
| 🟢 | control inventory › an ambiguous locator fails at once | 6.4s |
| 🟢 | control inventory › screenshot, restart, and clearState reopen the app on its home list | 11.5s |
| 🟢 | refusals › verbs a device never declares are refused before the engine | 26ms |
| 🟢 | refusals › keys the soft keyboard cannot send are refused by the engine | 911ms |
| 🟢 | refusals › a missing locator fails as not found once its timeout passes | 1.1s |
| 🟢 | tests/device-fixture.e2e.ts · 10 passed | 1m 36s |
| 🟢 | device fixture › setOrientation rotates the window the app measures | 6.4s |
| 🟢 | device fixture › home leaves the app, openApp brings it back where it was | 6.2s |
| 🟢 | device fixture › foregroundApp sees the home screen after home() | 5.9s |
| 🟢 | device fixture › closeApp then openApp relaunches on the home list | 13.6s |
| 🟢 | device fixture › closeApp leaves the next commands on the same device | 9.7s |
| 🟢 | device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN | 5.0s |
| 🟢 | device fixture › device.back pops the scenario | 5.5s |
| 🟢 | device fixture › unstable_raw reaches the test session through agent-device | 4.4s |
| 🟢 | device fixture › setNetwork and setAirplaneMode change what the app sees | 17.1s |
| 🟢 | device fixture › setLocation and clearLocation change what the app reads | 22.5s |
| 🟢 | tests/device.e2e.ts · 5 passed | 43.6s |
| 🟢 | permission prompt › allowing the system dialog enables the microphone | 6.7s |
| 🟢 | permission prompt › a denied prompt recovers once the permission is granted outside | 13.6s |
| 🟢 | sequential onboarding echoes the exact values | 10.4s |
| 🟢 | onboarding rejects a malformed email before advancing | 5.9s |
| 🟢 | dismissing the keyboard over choice controls leaves them as they were | 7.1s |
| 🟢 | tests/flows-lists.e2e.ts · 7 passed | 2m 9s |
| 🟢 | infinite scroll reaches the target item | 19.7s |
| 🟢 | product catalog adds exactly two of the right variant | 8.8s |
| 🟢 | gestures: long-press, then swipe left | 8.3s |
| 🟢 | gestures: double-tap completes the flow | 9.8s |
| 🟢 | sticky chrome: accept below the fold, then continue | 9.0s |
| 🟢 | async states: load, pull to refresh, claim through the toast | 11.8s |
| 🟢 | huge virtualized list reaches row 512 | 1m 2s |
| 🟢 | tests/flows.e2e.ts · 7 passed | 55.1s |
| 🟢 | text inputs unlock the submit button | 7.3s |
| 🟢 | filled text inputs report their values | 5.9s |
| 🟢 | modal flow confirms through the native alert | 9.8s |
| 🟢 | bottom tabs act inside the last tab | 7.2s |
| 🟢 | error recovery retries, then confirms the delete | 8.9s |
| 🟢 | choice controls place the exact order | 6.7s |
| 🟢 | debounced search selects the target once results arrive | 9.2s |
| 🟢 | tests/login-form.e2e.ts · 3 passed | 28.9s |
| 🟢 | login form › rejects a malformed email | 8.1s |
| 🟢 | login form › rejects an unknown account | 8.1s |
| 🟢 | login form › signs in and logs out | 12.7s |
| 🟢 | tests/smoke.e2e.ts · 1 passed | 6.4s |
| 🟢 | home lists the scenarios and opens one | 6.4s |
Tested by e2e 0.22.0 · 9m 22s · android-emulator · run artifacts
…eck clicks ARIA radios Match Playwright: inputValue() on a node with no value and isChecked() on a node with no checked state throw ACTION_FAILED. uncheck() clicks a role=radio; only a native radio refuses.
efcbbf2 to
3dc3030
Compare
|
[claude-opus-5-5] responding on behalf of okwasniewski Left for you (out of scope, listed in the body):
|
…s ARIA radios (#1114) * Throw on inputValue and isChecked of the wrong element, click ARIA radios on uncheck Ports e2e-inc/e2e#1552 (d2556bc). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls2e1yaZFgZ3FEJkYScfkK * Document uncheck on a native radio * Drop the radio special case Roles already covers Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls2e1yaZFgZ3FEJkYScfkK --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Follows the Playwright gaps plan: two items, both decided "match Playwright".
inputValue()on a node with no value (a heading, a button) throwsACTION_FAILEDwith Playwright's message, instead of"". Same ruletoHaveValuealready uses (isValueControlinpackages/e2e/src/internal/roles.ts): a value role with no value reported still reads"", since a device engine omits an empty value.isChecked()on a node with no checked state throwsACTION_FAILED("Not a checkbox or radio button"), instead offalse. Checkable means a reported state or one of Playwright'skAriaCheckedRoles(checkbox,menuitemcheckbox,option,radio,switch,menuitemradio,treeitem), so an ARIA switch withoutaria-checkedstill readsfalse.uncheck()on an ARIArole=radioclicks it, like Playwright. Only a native<input type=radio>refuses, with the same message as before. An ARIA radio whose click leaves it checked fails at the deadline with a hint about the radio group.ACTION_FAILEDbecause that is whatcheck()on a wrong element already gives.minorwith**Breaking:**. Docs updated:reference/screen.mdx,reference/errors.mdx,skills/e2ereads paragraph, and inmigrate/playwright.mdx(on top of docs(migrate): match the Playwright mapping to what e2e does #1545) thecheck()/uncheck(),inputValue(),isChecked()rows and thetoBeCheckednote.Marks removed (4
fail):check.e2e.ts"should uncheck radio by aria role", "should uncheck the box by aria role";locator-convenience.e2e.ts"inputValue should work", "isChecked should work".Callers checked: no agent, MCP, or engine code calls these reads; benchmarks read
textbox/switch/checkboxnodes; two integration/unit tests that pinned""/falseon a heading/listitem now assert the throw.Left as is, on purpose:
toBeCheckedstays lenient (.not.toBeChecked()passes on a button). Not in scope; the migrate guide says so.<label>, both reads now throw where Playwright reads the labeled control (before: a wrongfalse/""). Listed asmissingin the migrate guide.otherwith no value throws oninputValue(), astoHaveValuealready does there.Verified
Ran it locally: yes
node node_modules/e2e/dist/cli/bin.js run --tag playwright: 504 passed, 5 skipped, exit 0 (on main after fix(e2e): let setInputFiles([]) clear a file input #1546 and fix(web): elements inside a hidden iframe read as hidden #1549). The 4 unmarked tests pass, "should throw when trying to uncheck radio button" still passes.