Skip to content

Visual cursor overlay for input automation on headed Chrome #2401

Description

@ccl125

Is your feature request related to a problem? Please describe.

When an agent drives a real (headed) Chrome instance through chrome-devtools-mcp, mouse input is injected at the CDP level, so the OS cursor never moves. A human watching the browser cannot tell where the agent is clicking, which makes supervised sessions and live demos hard to follow.

Other agent harnesses (computer-use style tools) solve this by showing a visible cursor during automation, and it has become a commonly expected UX when agents operate a real browser.

Describe the solution you'd like

An opt-in --visual-cursor flag. When enabled, mouse-based input tools (click, dblClick, click_at, drag) animate a ghost cursor on the page before acting: the cursor smoothly slides to the target point and shows an expanding ripple at the click location. Animation duration is configurable via --visual-cursor-duration <ms> (default 800).

A working implementation exists in PR #2400 (closed pending this discussion): the cursor is injected via Page.addScriptToEvaluateOnNewDocument so it survives navigations, everything degrades silently so real input is never affected, and with the flag off (default) there is zero behavior or latency change.

I understand the concern about injecting scripts into inspected pages and am happy to rework the approach based on maintainer feedback - e.g. rendering the cursor inside an isolated world (Page.createIsolatedWorld) so page scripts cannot observe or interfere with it, or any other mechanism the team prefers.

Describe alternatives you've considered

  1. OS-level input automation that moves the real system cursor (computer-use style): heavyweight, platform-specific, requires extra OS permissions.
  2. Screencast recording: after-the-fact, not live.
  3. Agent-side page injection via evaluate: what many agents do today, but every harness reimplements it and it runs in the page's main world.
  4. Do nothing: humans keep guessing where the agent is clicking.

Additional context

Related: PR #2400 (initial implementation with tests and generated docs, closed with a request to file this issue first). This need came up while supervising an agent driving a real user Chrome via --autoConnect.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions