jackwener / jackwener/OpenCLI

[Bug]: click/keys/hover/drag report success but dispatch nothing when the page is not visible (background tab, minimized or occluded window)

Open
#2,520 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
29.3k
Forks
2.9k
Avg merge
15h 36m
Merged PRs (30d)
70

Description

## Summary

`browser click`, `hover`, `dblclick`, `drag` and `keys` return a success envelope while doing nothing at all whenever the session's page is not being composited by Chrome — a background tab, a minimized window, or a window that is fully occluded by another window (including a popup the page itself opened with `window.open`).

CDP `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent` resolve normally in that state, but Chrome never delivers the events to the renderer. Reads (`eval`, `extract`, `state`, `find`) and `type` (`Input.insertText`) keep working, so the session looks completely healthy and the agent has no way to notice that every write is being dropped.

This is a *silent* failure with a success-shaped return value:

```json
{"clicked": true, "target": "#toggleMenu", "matches_n": 1, "match_level": "exact"}
```

…while the page never sees the click.

## Environment

- opencli 1.8.6 (also reproduced against a build of `upstream/main` @ `8271afc6`, v1.8.8)
- Browser Bridge extension v1.0.22
- macOS 15 (Darwin 25.6.0), Chrome, Node v22.22.0

## Minimal reproduction

`index.html`, served with `python3 -m http.server 8765`:

```html

opencli focus repro
#menu{display:none}#menu.open{display:block}
open a popup (window.open)
toggle dropdown

item A

marker: none

let n = 0;
document.getElementById('openTab').addEventListener('click', () => window.open('popup.html', 'reproTab'));
document.getElementById('toggleMenu').addEventListener('click', () => {
document.getElementById('menu').classList.toggle('open');
document.getElementById('marker').textContent = 'CLICKED-' + (++n);
});

```

```bash
opencli browser repro open "http://127.0.0.1:8765/index.html"

# 1) baseline — page is visible, the click lands
opencli browser repro eval "document.visibilityState" # visible
opencli browser repro click "#toggleMenu" # {"clicked": true, ...}
opencli browser repro eval "document.getElementById('marker').textContent"
# CLICKED-1 ✅

# 2) let the page open a popup, so the session's window is no longer the
# composited one (any other way of hiding the window works too: minimize
# Chrome, or switch to another tab in the same window)
opencli browser repro click "#openTab"

# 3) same click, same selector, same session
opencli browser repro eval "document.visibilityState" # hidden
opencli browser repro click "#toggleMenu" # {"clicked": true, ...}
opencli browser repro eval "document.getElementById('marker').textContent"
# none ❌ the listener never ran
```

`keys` fails the same way, and just as quietly:

```bash
opencli browser repro eval "window.__k=0; document.addEventListener('keydown', () => window.__k++); 'ok'"
opencli browser repro keys Enter # prints "Pressed: Enter"
opencli browser repro eval "String(window.__k)"
# 0 ❌
```

Meanwhile the DOM path works perfectly on the very same hidden page:

```bash
opencli browser repro eval "document.getElementById('toggleMenu').click(); document.getElementById('marker').textContent"
# CLICKED-1 ✅
```

## Expected vs actual

**Expected:** either the input reaches the page, or the command reports that it could not be delivered.

**Actual:** `{"clicked": true}` for a click the page never received. An agent driving a multi-step flow keeps going and every subsequent step is built on a state that never happened.

## Workaround

Make the session's page the composited one again and the identical commands start working:

- background tab in the same window → `opencli browser tab select `
- occluded or minimized window → raise/restore the Chrome window by hand

There is no way to detect the bad state from a command's output, so an agent cannot know when to apply the workaround.

## Root cause

At `8271afc6`:

- `src/browser/base-page.ts:327` — `click()` commits to the native path when the measured rect is non-zero and the point hit-tests onto the target. Both of those are computed with `getBoundingClientRect()` / `elementFromPoint()`, which keep returning entirely correct values on a hidden renderer, so nothing in the probe notices the problem.
- `src/browser/base-page.ts:353` — `tryNativeClick()` returns `true` as soon as the CDP call resolves. `nativeClick()` (`src/browser/base-page.ts:1247`) only issues `Input.dispatchMouseEvent`, and Chrome resolves it whether or not the renderer consumes it. The `el.click()` fallback below it is therefore never reached.
- `src/browser/base-page.ts:1017` — `pressKey()` has the identical shape: `tryNativeKeyPress()` resolving is taken as proof the key was delivered, so `pressKeyJs()` never runs.
- `extension/src/background.ts:2023` — `handleCdp()` attaches the debugger and forwards `Input.*` without ever checking or changing the tab's activation/visibility. The only code that activates a tab is `handleTabs`' `select` case (`extension/src/background.ts:1940`), i.e. the `tab select` workaround.

So every layer reports success and none of them observes delivery.

## Two related observations

1. `browser screenshot` (`Page.captureScreenshot`) forces a compositor frame, and immediately afterwards CDP input starts landing on the same still-`hidden` page. That makes the bug look intermittent if a screenshot happens to sit between two clicks.
2. `tab list` reports `active: false` for the session's page in the background-tab case, so the extension does have the signal — it just isn't consulted on the input path, and it does not cover the minimized/occluded-window case (where the tab is `active: true` and the renderer is still hidden).

## Related

- #2418 — same "click reports `clicked: true` while dispatching nothing" shape, from CLI/extension version skew rather than visibility. Different trigger, same missing delivery check.
- #2076 — led to `click_method` being surfaced; the same idea applies here, since the JS path is the one that actually works while hidden.
- #2479 — the lifecycle RFC notes that automation should stay out of the user's way; this is the failure mode that makes "stay out of the way" silently lossy today.

I have a fix and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

Research direction

Start with src/browser/base-page.ts at click(), tryNativeClick(), nativeClick(), pressKey(), and tryNativeKeyPress(), then inspect extension/src/background.ts at handleCdp() and the handleTabs select case. Reproduce the provided index.html scenario with a hidden or occluded page; done means click and key commands either reach the page or report that delivery failed instead of returning false success.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli, web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.