Windows Computer Use: repeated window-state recovery and human-input handoff overhead

What version of the Codex App are you using (From “About Codex” dialog)?

26.901.51231

What subscription do you have?

pro 20x

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

During an authorized Windows desktop workflow to configure PDF translation in Zotero, Codex repeatedly performed window discovery, state capture, activation, recapture and retries.

The task eventually completed, but took 51 minutes 12 seconds and 117 outer tool calls, including 50 GUI-related calls. These counts represent tool wrappers, not individual clicks.

Observed errors included:

• "coordinate input geometry is unavailable"
• "element [id] is not available in cached app state for [application]"
• "foreground window did not report a process id"
• "user input was detected in this window; call get_window_state before continuing"
• A UIA CacheRequest property error (0x80070057).

Expected behavior:
Reliable target/state validation and efficient recovery after window changes or human intervention, while preserving human-input safeguards and permission checks.

This is one observational run, not a controlled reproduction. Some additional failures were agent scripting, dependency, encoding or download issues. The report distinguishes these from GUI runtime errors and does not attribute all delays to the network or runtime.

The accompanying sanitized evidence package includes a relative-time timeline, metrics, 13 curated error occurrences, analysis, optimization suggestions and a proposed reproduction plan. It excludes credentials, screenshots, document content, private paths and original session identifiers.

OpenAI-Computer-Use-Feedback-Sanitized.zip

What steps can reproduce the bug?

Reproducibility:
Observed during one real workflow. A deterministic minimal reproduction has not yet been established.

Observed workflow:

1. Use the Codex desktop app on Windows with the Computer Use plugin.
2. Ask Codex to configure a PDF translation workflow in Zotero 10.0.1 using PDF2zh 4.1.7, preserving the original PDF.
3. Let the agent navigate plugin and settings dialogs, inspect window state, activate windows and edit configuration.
4. During this run, the runtime detected human input and required a fresh window-state observation.
5. Inspect the tool results and recovery sequence.

Observed results:
Repeated window discovery, state capture, activation and recapture occurred. Errors included unavailable coordinate geometry, unavailable cached elements, foreground process identification failure and a UIA CacheRequest property error.

The task eventually completed in 51m12s with 117 outer tool calls, including 50 GUI-related calls. This does not establish that every repeated observation was unnecessary or that human input caused all failures.

Evidence:
The sanitized ZIP contains the relative-time timeline, reviewed errors and a proposed synthetic reproduction plan. That proposed plan has NOT been executed.

Session ID: omitted from this public report for privacy.
Token limit and context usage: not established for this report.
No standalone reproduction code is currently available.

What is the expected behavior?

No response

Additional information

No response

Start with the sanitized evidence package’s relative-time timeline, curated error occurrences, metrics, and proposed reproduction plan. Establish a deterministic minimal reproduction for the Windows Computer Use workflow and separate GUI runtime failures from agent scripting, dependency, encoding, and download issues; done means the recovery behavior and remaining overhead are measured against that reproduction.