feat: gate experimental pointer hover action - #658
Conversation
…mentalPointerActions Implement the maintainer's preferred scope for alibaba#222: expose a single experimental pointer-based action (hover) only when experimentalPointerActions: true is set on both PageControllerConfig and AgentConfig. Disabled by default so the tool surface stays minimal until real usage shows the value. Mirrors the existing experimentalScriptExecutionTool pattern: - PageController.hoverElement(index) returns an explanatory failure when the flag is off, and dispatches pointerover/pointerenter + mouseover/mouseenter otherwise. - PageAgentCore deletes hover_element_by_index from this.tools at construction unless experimentalPointerActions is true. Extension bridge is intentionally untouched. Tests cover tool gating in PageAgentCore and the hoverElement method behavior in PageController.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebc834a84f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
ebc834a to
599b494
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 599b494a71
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52998961f5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Addressed the current review feedback in
Validation: page-controller 8/8, core 22/22, and @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7cb8ba37d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Follow-up for the extension-path review feedback: Page-controller 8/8 and core 22/22 pass; the extension typecheck remains blocked by the repository's existing WXT/CSS/simple-icons environment errors. @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dd3281ee7b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Addressed the two latest P2 findings in
Validation is now page-controller 9/9, core 22/22, and @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f634c55e61
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Addressed the latest P2 in
Validation is now page-controller 10/10, core 22/22, and @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb78116e72
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Addressed the latest P2 in
Validation is now page-controller 11/11, core 22/22, and @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 602c4f34d6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const elemText = this.elementTextMap.get(index) | ||
| await hoverElement(element, this.syntheticHoverPath) | ||
| const retained = this.syntheticHoverPath.filter((hovered) => hovered.contains(element)) | ||
| this.syntheticHoverPath = retained.includes(element) ? retained : [...retained, element] |
There was a problem hiding this comment.
With the new ancestor-enter dispatch, a direct hover on a child now fires mouseenter/pointerenter on wrappers, but this state update still records only the target when there was no previously retained ancestor. In that scenario, the next click or hover outside calls clearSyntheticHover with a path containing only the child, so native non-bubbling leave listeners on the wrapper never run and JS-opened menus/tooltips can remain stuck open; store the full entered ancestor path, not just the indexed element.
Useful? React with 👍 / 👎.
|
Addressed the latest P2 in
Validation: page-controller 12/12, core 22/22, and @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 739cf43b16
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| this.assertIndexed() | ||
| const element = getElementByIndex(this.selectorMap, index) | ||
| const elemText = this.elementTextMap.get(index) | ||
| this.syntheticHoverPath = await hoverElement(element, this.syntheticHoverPath) |
There was a problem hiding this comment.
When hover_element_by_index is the last action in a task, or the run is stopped before a later click/input/select, the synthetic hover path stored here is never transitioned out: the agent’s normal teardown only cleans highlights/mask, and dispose() now drops the path without dispatching leave events. For JS-controlled menus/tooltips that close in mouseleave/mouseout, this leaves the page in a hovered/open state for the user or the next task until some later pointer action happens; clear the synthetic hover with leave/out events during task/controller cleanup as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3e015863c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| if (!movingWithinPreviousElement) { | ||
| previousElement.dispatchEvent( | ||
| new PointerEvent('pointerleave', { bubbles: false, relatedTarget }) |
There was a problem hiding this comment.
When a component opens/closes hover state in pointer handlers that filter for event.pointerType === 'mouse', the synthetic enter path supplies pointerType: 'mouse', but this leave path creates pointerleave without it. Browser mouse leave events carry the mouse pointer type; here the synthetic event defaults to an empty pointer type, so those close handlers can ignore the cleanup and leave JS-opened menus/tooltips stuck after the agent moves or clicks away. Include the same pointer metadata on the synthetic pointerout/pointerleave events.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4cf203d6ae
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| this.assertIndexed() | ||
| const element = getElementByIndex(this.selectorMap, index) | ||
| const elemText = this.elementTextMap.get(index) | ||
| this.syntheticHoverPath = await hoverElement(element, this.syntheticHoverPath) |
There was a problem hiding this comment.
When an agent clicks an element that opened hover-driven UI and then calls hover_element_by_index on a different, non-contained element before another click/input, the clicked element never receives the synthetic pointerleave/mouseleave that clickElement() defers via its module-level lastClickedElement. This path only transitions syntheticHoverPath, so a menu or tooltip opened by the click action's hover events can stay open and interfere with the subsequent hover/observation until a later click happens.
Useful? React with 👍 / 👎.
|
Addressed the two latest Codex P2 findings in
Validation:
The repository-wide typecheck remains blocked by the existing extension |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8ad0772dea
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| await movePointerToElement(element, x, y) | ||
|
|
||
| const pointerOpts = { |
There was a problem hiding this comment.
When the indexed target is inside a web component shadow root (the DOM walker already descends into node.shadowRoot), these constructed pointer/mouse events default to composed: false, so the bubbling pointerover/mouseover events never cross to the host or document. Components that attach hover handlers on the host will not open/close even though hover_element_by_index reports success; include composed: true for the synthetic hover/leave events that should escape the shadow boundary.
Useful? React with 👍 / 👎.
1 participant
Summary
experimentalPointerActionsconfiguration to the agent and page controllerhover_element_by_indexonly when the experimental flag is enabledrelatedTargetand without premature leave events:hoverRelated to #222.
Validation
npm test --workspace=@page-agent/page-controller(11 tests)cd packages/core && npx vitest run --config vitest.config.js(22 tests)git diff --checknpx tsc --noEmit -p packages/extension/tsconfig.jsonwas attempted but is blocked by existing repository environment issues (.wxt/tsconfig.json, WXT globals, CSS module declarations, andsimple-iconstype resolution); no errors were reported in the changed extension controller files.The change is intentionally opt-in and does not alter the default tool surface.