apache / apache/maka

Agent cursor sinks behind covering windows during Computer Use

Open
#5,048 1 comment 0 reactions 1 assignee Claimed by @wutongyuonce View on GitHub
Dominant language
TypeScript
Stars
5.4k
Forks
502
Avg merge
1d 2h
Merged PRs (30d)
706

Description

### Problem

maka-cu already computes `obscuringRects` for windows stacked above the Computer Use target. The host stores them on the observation record. `CuObservation` even documents that the agent cursor should test the point it is about to draw — a control on the top edge can be visible while the middle of the window is buried. The overlay hook never receives those rects. `keepElevated` is true only when there is no target window id, so a covered aim point still sinks into the target window z-order and can disappear under the covering window.

### Desired outcome

When the cursor aim point falls inside an obscuring rect, it stays elevated. When the point is clear and a target window id exists, it still sinks (so it does not float over the user's other windows). Missing or empty rects keep today's sink-when-window-id-exists behaviour. Elevation is decided per point, not per whole window.

### Alternatives or workarounds

Always staying elevated. That was already rejected: it put the cursor over the user's own windows.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.