ChromeDevTools / ChromeDevTools/chrome-devtools-mcp

Visual cursor overlay for input automation on headed Chrome

Open
#2,401 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

collecting-feedback feature
Dominant language
TypeScript
Stars
52.3k
Forks
4.3k
Avg merge
2d 7h
Merged PRs (30d)
83

Description

Is your feature request related to a problem? Please describe.

When an agent drives a real (headed) Chrome instance through chrome-devtools-mcp, mouse input is injected at the CDP level, so the OS cursor never moves. A human watching the browser cannot tell where the agent is clicking, which makes supervised sessions and live demos hard to follow.

Other agent harnesses (computer-use style tools) solve this by showing a visible cursor during automation, and it has become a commonly expected UX when agents operate a real browser.

Describe the solution you'd like

An opt-in --visual-cursor flag. When enabled, mouse-based input tools (click, dblClick, click_at, drag) animate a ghost cursor on the page before acting: the cursor smoothly slides to the target point and shows an expanding ripple at the click location. Animation duration is configurable via --visual-cursor-duration <ms> (default 800).

A working implementation exists in PR #2400 (closed pending this discussion): the cursor is injected via Page.addScriptToEvaluateOnNewDocument so it survives navigations, everything degrades silently so real input is never affected, and with the flag off (default) there is zero behavior or latency change.

I understand the concern about injecting scripts into inspected pages and am happy to rework the approach based on maintainer feedback - e.g. rendering the cursor inside an isolated world (Page.createIsolatedWorld) so page scripts cannot observe or interfere with it, or any other mechanism the team prefers.

Describe alternatives you've considered
  1. OS-level input automation that moves the real system cursor (computer-use style): heavyweight, platform-specific, requires extra OS permissions.
  2. Screencast recording: after-the-fact, not live.
  3. Agent-side page injection via evaluate: what many agents do today, but every harness reimplements it and it runs in the page's main world.
  4. Do nothing: humans keep guessing where the agent is clicking.
Additional context

Related: PR #2400 (initial implementation with tests and generated docs, closed with a request to file this issue first). This need came up while supervising an agent driving a real user Chrome via --autoConnect.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the closed PR #2400, including its tests and generated documentation, then trace the click, dblClick, click_at, and drag input tools and the existing CLI flag parsing. Confirm how --visual-cursor and --visual-cursor-duration should behave, including navigation, silent degradation, and the default path. Done means the opt-in animation works for those tools without changing input behavior or adding latency when disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, devtools, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.