openai / openai/codex-plugin-cc
feature request: /codex:computer-use to delegate runtime/UI verification (app launch, screenshots, simulators)
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 33.3k
- Forks
- 2.3k
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The plugin has two delegation shapes today: reviewing code (/codex:review, /codex:adversarial-review: read-only, git-scoped, structured findings) and changing code (/codex:rescue: write-capable, free-form). Neither answers the question that comes right after an implementation lands: does it actually work at runtime?
Codex has computer-use capability, but the plugin exposes no path to it. Verification is its own job with its own requirements. It needs to be read-only on source, it needs evidence artifacts (screenshots, logs), it needs a pass/fail/blocked verdict, and it often needs GUI-level access. Prompting codex-rescue to "test the app" fights that agent's write-capable defaults and thin-forwarder design, and returns no structured verdict. #354 describes the same category error for review: bending the wrong tool with prompt instructions.
Use cases
- Post-implementation UI check. Claude finishes a signup form, then
/codex:computer-use verify the signup flow on localhost:3000, happy path plus validation errors --wait. Codex opens the browser, walks the flow, screenshots each step, and returns pass or fail with evidence paths. - Background verification while work continues. Codex verifies feature A with
--background(results via the existing/codex:statusand/codex:result) while Claude implements feature B. - Environments Claude can't reach: iOS simulator, Electron app, desktop window state. A second agent that didn't write the code doesn't inherit the implementer's assumptions.
Proposed shape
Everything below is a sketch to anchor discussion; the design is yours. The idea is to mirror the review architecture so the existing runtime does most of the work.
/codex:computer-use [--wait|--background] [--full-access] [what to verify, how to launch, what counts as pass]- A companion subcommand with the same foreground/background execution model as
review, with jobs visible tostatus,result, andcancel. - A verification prompt template under
prompts/that requires a self-contained brief: behavior to verify, platform and app type, launch commands and test credentials, whether source edits are allowed (default: no), artifact paths, and the return contract. - A structured output schema under
schemas/:verdict: pass|fail|blocked,steps_performed[],observed_behavior,screenshots[],findings[],confidence. - A runtime-managed artifact directory per job for screenshots, logs, and the report.
- Possible v2: a model-invocable
codex-verifiersubagent so Claude can dispatch verification proactively. That intersects #269/#238 and #354, so command-only is a fine v1.
Safety defaults
- Source edits forbidden by default; an unexpected diff is a finding, not a success.
- Sandbox tiers:
workspace-writefor non-GUI runtime checks. GUI automation (app launch, simulators, screenshots) sits behind an explicit opt-in. #145 already requests the same--full-accessflag fortask; this feature would be its second consumer. - Disruption boundary: launching apps, simulators, or browsers to verify the requested work is within the job's mandate. Closing the user's apps, changing system settings, or acting on real accounts or data requires explicit consent in the brief.
- No auto-fix after a
failverdict. The existing result-handling rule applies unchanged.
Fallbacks
- Codex missing or unauthenticated: stop and point to
/codex:setup, same as every other command. - Headless environment, no display, simulator absent: return
verdict: blockedwith the reason instead of hanging. - Long runs (app boots and simulators are slow): background mode is the primary path, and the Bash 10-minute-ceiling work in #370 applies here too.
- Model or account without computer-use capability: a specific error instead of a generic API failure.
Alternatives considered
- Prompting
codex-rescue: wrong defaults (write-capable), no verdict schema, no artifact contract, and its own guidance forbids verification-shaped wrapper work around the run. - Claude-side browser automation (Playwright MCP and similar): useful, but not independent, since the implementer verifies itself. It also occupies Claude's turn instead of running in parallel.
- A local personal skill: works today, and it is where this idea comes from (see Inspiration). But every user then reinvents prompt contracts, artifact handling, sandbox policy, and the cross-platform process management this plugin's runtime already solves centrally.
Inspiration
I got the idea after watching Theo's (t3.gg) "A proper guide to Fable 5" (https://www.youtube.com/watch?v=8GRmLR__OGQ, from ~23:05), where he shows a hand-rolled codex-computer-use skill next to sibling skills for Codex-delegated review and implementation. His version defines the verification stance, a do-not-use boundary (not for ordinary code reading, typechecking, linting, or tests Claude can run directly), a 4-step workflow around codex exec, and a prompt-requirements checklist. It works as a personal skill, but this capability makes more sense living centrally in the official plugin, where the runtime, job management, and sandbox policy already exist, than as something each user rebuilds for themselves.
Together with #354 (reviewer subagent) and #263 (/codex:implement), this would complete the set: review, implement, verify.
Duplicate check
Searched titles, bodies, and comments across all open and closed issues and PRs for "computer use", "computer-use", screenshot(s), browser automation, simulator, playwright, "ui test", "visual verification", "verify ui". No existing request; all hits were incidental. Closest in spirit are the image-generation PRs #357/#271, which are a different capability.
Open questions
- Platform phasing: is macOS/Linux-first acceptable, given the Windows process/spawn complexity already in the tracker?
- Screenshot artifact retention: prune with job state, keep the last N, or leave it to the user?
- Naming:
/codex:computer-use(mechanism, matches OpenAI terminology) vs/codex:verify(intent, matches the review/rescue verb family). No strong attachment.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing review command and its foreground/background job flow, including /codex:status, /codex:result, and /codex:cancel. Then inspect the prompts/ and schemas/ areas and the runtime artifact handling described in the issue. Done means a command-only verification path with explicit safety defaults, structured pass/fail/blocked results, evidence artifacts, and the documented fallback behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- cli, developer-experience, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100