Desktop app: annotate the in-app preview and attach the marked image to the active task

Open
#35,502 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
rust
Domain
desktop

Research direction

Start by locating the Codex desktop app's in-app preview pane and preview toolbar entry points. Define the first-release flow around freehand drawing, undo, clear, capture, and attach-to-task, then verify that marks do not interact with the inspected page and that the bounded image appears in the active task timeline.

Written by the indexing model from the issue text.

Description

app browser enhancement

Problem

Visual review in the Codex desktop app is slower than it needs to be when a user is looking at the built-in preview pane. In a voice or live-review flow, the user can point at a part of the page, but Codex cannot see the pointer or receive a precise visual reference.

Requested capability

Add an annotation mode to the in-app preview pane:

  • A visible annotation control in the preview toolbar.
  • Mouse or pen drawing for circles, arrows, and freehand marks directly over the preview.
  • A capture action that produces an annotated screenshot.
  • One-click attachment of that image to the active Codex task, with an optional short note.
  • The annotation is non-destructive: it overlays the preview and never changes the inspected webpage.

Why it matters

This closes the gap between visual review and agent execution. It lets a user communicate "this exact element" without hunting for selectors, describing layout positions in words, or leaving the Codex app.

First release scope

A compact first version could support freehand drawing, undo, clear, and attach-to-task. Later versions could add arrows, rectangles, text notes, and annotations that persist alongside the task timeline.

Acceptance criteria

  1. A user can enter and exit annotation mode from an in-app preview.
  2. Marks are drawn over the current preview without interacting with the underlying page.
  3. The user can undo or clear marks before capture.
  4. Capturing attaches a bounded-size image to the current task as visual context.
  5. The task can continue with the annotated image visible in its timeline.

This is particularly useful for voice-driven site and UI review, where a user can say "circle this" and immediately give the agent an unambiguous visual target.

Dominant language
Rust
Stars
125k
Forks
19.5k
Avg merge
1m
Merged PRs (30d)
1k

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openai/codex

All issues in openai/codex

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.