zcode-cua 0.5.12 (Windows): all raw coordinate clicks refused with 'dispatch app/window identity does not match frame'
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 22
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
zcode-cua 0.5.12 (Windows): every raw coordinate click refused — "dispatch app/window identity does not match frame"
Summary
On Windows 10, the zcode-cua plugin refuses 100% of raw coordinate pointer actions (left_click and the other pointer tools with a coordinate target), for both window-bound rasters and full-display rasters, regardless of focus, overlays, or window position. Accessibility-element actions, app-scoped keyboard, screenshots and enumeration all work fine, so the plugin remains usable — but pixel-precise clicking (canvas-like surfaces such as Paint) is impossible.
Environment
- OS: Windows 10 Pro, build 19045 x64, single display 1920x1080
- Plugin:
zcode-cua0.5.12 (official marketplacezcode-plugins-official) - Broker: "ZCode Computer Use" helper, backend
electron, apiZCodeComputerUseIPC-1 - Behavior is identical before and after a full ZCode desktop-app restart (helper pid 24384 → 29748), which rules out client/helper version skew within a session
request_accessreports: accessibility granted, screen recording granted,raw_mouse_keyboard: true,coordinate_hit_test: true
Symptom
Every pointer action with a coordinate target returns action_sent=false:
- window-bound frame:
click: dispatch app/window identity does not match frame <frame_id> - full-display frame:
click: full-display frame <frame_id> is stale; the live pixel owner changed before dispatch
Tried and refused in all combinations:
strategy: "auto"andstrategy: "event"- with and without explicit
frame_id(i.e. transport-local current raster too) - with and without
app_ref(plain pid, and pid +window_idpinned) - on freshly captured rasters (
get_app_stateimmediately followed by the click) - on multiple targets (classic Win32 app
mspaint, desktop background) open_application(activate=true)before clicking (reportsactive:true)
Isolation performed (common causes ruled out)
- Invisible full-screen overlays. The machine had a phantom full-screen
LockApp.exewindow (Windows Spotlight leftover, 1920x1080) plus two invisible full-screenTextInputHost.exewindows. Killed both processes, verified gone → refusals unchanged. - Foreground/focus. Target window forced to true OS foreground via minimize→restore +
SetForegroundWindow; verifiedGetForegroundWindow() == target HWNDimmediately before the click → refusal unchanged. (list_windowsfocusedflag also true.) - Child-HWND ambiguity. Clicks aimed at the title bar (the same HWND the frame was captured from, no child-window ambiguity) are refused identically → not a child-window hit-test mismatch.
- Frame staleness / timing. Fresh
get_app_stateraster followed by an immediate click → refused. Same result after the full ZCode restart. - Target app. Reproduced on
mspaint(classic Win32) and against the desktop background → not app-specific.
Minimal repro
open_application({"name":"mspaint"}, activate=true)get_app_state(pid, include_screenshot=true)left_click({"type":"coordinate","x":X,"y":Y})using coordinates from the returned raster- →
click: dispatch app/window identity does not match frame <frame_id>— every time
What works (same session, same apps)
get_app_state(incl. screenshots),screenshot,list_apps,list_windows,request_access- Element-target actions:
left_click(AXPress) on ribbon buttons and other UIA elements typeinto editable element targets;key/hold_key/typewithapp_ref— app-scoped keyboard reaches the target app even when it is not the OS-foreground windowwrite_clipboard/ PowerShell clipboard interop
Workaround used in practice: drive UI via accessibility elements and app-scoped keyboard; for Paint's canvas, render text to a bitmap, place it on the clipboard and paste (Ctrl+V), then commit via a ribbon element press.
Suspected area
The dispatch-time identity verification for coordinate targets on the Windows Electron backend appears to never accept its own freshly captured frames — possibly a DPI/coordinate-transform mismatch or a frame→HWND identity mapping issue in the helper. The same plugin is reported working on macOS (CGWindowID-based), so this looks Windows-specific.
Happy to run further diagnostics on the affected machine if useful (happy to test dev builds/patched helpers too).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the minimal repro: open_application, get_app_state(include_screenshot=true), then left_click using coordinates from the returned raster. Inspect the Windows Electron backend's dispatch-time identity verification; done means coordinate-target clicks are accepted for freshly captured window-bound and full-display frames on the described Windows cases while existing element actions remain working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- electron
- Domain
- desktop-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100