Windows Computer Use cannot reliably target or type into an ArcGIS Pro owned dialog
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What version of the Codex App are you using (From “About Codex” dialog)?
26.908.4834.0 (installed desktop package version)
What subscription do you have?
Pro
What platform is your computer?
Windows build 26200.9457, display version 25H2
What issue are you seeing?
- The main Pro window is the only target returned by both inventories. The owned New Script dialog is visible in screenshots and in the accessibility tree.
- Parent captures are 897x814 and 922x827 logical pixels. The actual dialog capture is 1435x700, with a separate 1456x726 shadow capture, and extends beyond the parent.
- Clicking Name through the full dialog screenshot repeatedly times out around 10.1 seconds. Before restart, an input attempt also changed the parent from maximized to restored size.
- A Cancel click through the full dialog screenshot failed with "failed to activate captured window".
- Clicking the indexed Cancel element after fresh activation/state returned: "point (1237, 793) is outside window bounds { originX: 0, originY: 0, width: 922, height: 827 }".
- Clicking Name through its visible portion in the parent capture succeeded and produced a visible caret. Subsequent Control+A timed out; set_value timed out in 10,063 ms; type_text once acknowledged success without visible text change.
- After resetting the official JavaScript control session, a fresh parent-capture click again established a caret, but type_text timed out in 10,108 ms. Name remained Script.
- Escape and Alt+F4 acknowledged success without observed dialog closure in the earlier session. A later attempt to move the dialog for mouse-only closeout also timed out.
- Restarting Pro into a new process and a new empty temporary toolbox did not fix it. No script tool was saved or run.
Working comparison:
In the same investigation, a disposable Paint Save As dialog accepted the filename through type_text, Cancel worked, and the unchanged Paint window closed normally. Ordinary dialog input can work; a single targetable parent or null accessibility alone does not explain every failure.
What steps can reproduce the bug?
- Restart ArcGIS Pro and choose Start without a template.
- In Catalog, expand Toolboxes, right-click the empty temporary Default.atbx, and choose New > Script.
- Select the unique Pro Window returned by sky.list_windows(). Request get_window_state with screenshots and accessibility text.
- On the fresh full New Script dialog screenshot, click the Name field using that screenshot's ID and coordinates measured on that image.
- Reobserve before subsequent actions. Compare a click on the same visible Name control within the parent screenshot, then attempt text entry after an observed caret.
- Test the identified Cancel element after fresh state. Check whether list_windows or list_apps offers a separate Window for New Script.
No customer project or script execution is needed.
What is the expected behavior?
Preserve target validation while resolving an observed owned dialog's actual input window and bounds. Expose a separate supported Window when necessary. Verify screenshot-coordinate and indexed-control targeting and whether parent activation redirects or prevents keyboard delivery. A successful action acknowledgment should agree with the observed result.
The installed JavaScript client defaults ordinary requests to 10 seconds. The documented action API has no timeout option. Increasing only the outer JavaScript timeout has not been a remedy, and no internal client option was substituted.
Acceptance should include entering and verifying a harmless Name value, restoring it, navigating another property page, Cancel, then repeating after reopening the dialog without manual focus repair. The implementation cause is not confirmed.
Additional information
Bundled Computer Use skill 26.908.40834; @oai/sky 0.6.6. ArcGIS Pro executable product/file version 3.7.1.1904. Documented node_repl and sky window2 API only; no patched runtime, alternate automation client, or permission bypass.
Related, but not established as the same cause: https://github.com/openai/codex/issues/44104 (an observable but untargetable WeCom dialog).
The redacted reproduction was also submitted to OpenAI Support. Local usernames, business paths, customer project artifacts, screenshots, credentials, and conversation transcripts are excluded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the ArcGIS Pro New Script dialog case using the documented JavaScript client and sky.list_windows(), get_window_state, screenshots, accessibility text, and indexed controls. Compare parent-capture and full-dialog targeting, then verify that Name entry, Cancel, reopening, and observed action results behave consistently without manual focus repair.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, rust
- Domain
- desktop, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100