openai / openai/codex

Windows Computer Use: Filmora File-menu popup capture returns underlying window pixels

Open
#45,080 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app bug computer-use windows-os
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Summary

Windows Computer Use detects a Filmora File-menu transient surface, but its returned image contains underlying Media/timeline pixels instead of the menu. This blocks reliable menu observation and a supervised editing pilot. Please route to the Computer Use capture owner if this tracker is not the right component.

Environment

  • Windows desktop (exact OS build not collected)
  • OpenAI Computer Use plugin: 26.908.40834
  • Wondershare Filmora: 15.7.25.21914
  • Codex desktop app version: not independently collected
  • Observation API: sky window2 get_window_state

Reproduction already observed in the editing task

  1. Bind to the Filmora main window and capture its screenshot/accessibility state. Ordinary main-window capture works.
  2. Click the observed File menu.
  3. Capture the state with screenshot and accessibility text enabled.
  4. A second transient image is returned, but it shows underlying Media/timeline pixels rather than File-menu contents.
  5. Clicking File using its observed accessibility target reproduced the same symptom as coordinate targeting.

Expected

The transient screenshot should contain the actual File-menu pixels, with a correct surface/rectangle mapping; accessible menu entries would also permit reliable observation.

Actual evidence

  • Main image: 1707 x 1019, origin (0, 0).
  • Detected popup image: 265 x 574, origin (261, 47), zIndex 1.
  • Popup content is an unrelated crop of the underlying main window.
  • Accessibility exposes top-level menu labels, but not popup entries.
  • Window enumeration did not expose a separately targetable popup.
  • Fresh capture and one supported rebind/activation recovery reproduced the problem.
  • Escape dismissed the popup and subsequent capture returned one normal main-window image.
  • The documented API available in that task exposes window/include_screenshot/include_text, not an alternate capture backend or popup-surface selector.

Scope and uncertainty

This report summarizes the executing assistant's direct observations; no new reproduction was performed to file it. A transient-surface capture/composition problem is suspected, but the internal root cause is NOT proven. The evidence does not establish whether this affects other applications or menus.

No clip placement, trim, delete, speed adjustment, save, or media mutation was performed during the diagnosis. The pilot remains stopped. Private project titles, source routes, screenshots and full conversation logs are intentionally omitted.

Requested investigation

Please investigate transient-popup source-surface/rectangle composition and/or popup accessibility exposure. A fix should be verified by comparing the actual visible File menu with the returned popup image, then checking dismissal restores normal capture.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the implementation of the “sky window2 get_window_state” observation entry point and trace how transient surfaces are captured and mapped. Reproduce with a Filmora File menu using screenshot and accessibility text enabled, then compare the returned popup image with the visible menu and confirm that dismissal restores normal capture.

Written by the indexing model from the issue text.

Assessment

Domain
desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.