openai / openai/codex

`codex exec review` accepts but ignores `--output-schema`

Open
#38,545 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI code-review exec
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

Component

Codex CLI bundled with the ChatGPT desktop app for macOS.

Version
  • Reproduced on codex-cli 0.148.0-alpha.9
  • Previously reproduced on 0.147.0-alpha.6.6
  • macOS 26.6.1 (arm64)
Summary

codex exec review accepts and advertises --output-schema, but the review command ignores the schema and writes its normal prose review to --output-last-message. The command exits 0, so automation cannot distinguish this from successful schema enforcement.

Ordinary codex exec honors the same schema.

Minimal reproduction

Create a repository containing a commit with an obvious defect:

# example.py
def divide(a, b):
    return a / 0

Use this schema:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "additionalProperties": false,
  "required": ["verdict"],
  "properties": {
    "verdict": {
      "type": "string",
      "enum": ["pass", "block"]
    }
  }
}

Run:

codex exec review \
  --ephemeral \
  --ignore-user-config \
  -m gpt-5.6-terra \
  -c 'model_reasoning_effort="low"' \
  --commit HEAD \
  --output-schema minimal-schema.json \
  -o review.txt
Actual result

The command exits 0 and review.txt contains prose:

The newly added function cannot successfully divide any inputs because it unconditionally divides by zero.

Review comment:

- [P1] Divide by the supplied denominator — .../example.py:2-2
  This always evaluates a / 0 ...

The response does not match the supplied schema.

Control

The same schema works with ordinary exec:

codex exec \
  --ephemeral \
  --ignore-user-config \
  -m gpt-5.6-terra \
  -c 'model_reasoning_effort="low"' \
  --output-schema minimal-schema.json \
  -o control.txt \
  'Return verdict pass. Do not inspect files or call tools.'

control.txt contains:

{"verdict":"pass"}
Expected result

Either:

  1. codex exec review forwards --output-schema to the review turn and guarantees that the final message matches it, or
  2. the review subcommand rejects --output-schema as unsupported instead of accepting it as a silent no-op.
Impact

This makes exec review unsafe for machine-readable CI gates: the documented flag appears to provide schema enforcement, but automation receives unstructured prose with a successful exit code.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the codex exec review entry point and compare its --output-schema handling with ordinary codex exec, using the minimal reproduction and control commands in this issue. Trace whether the schema reaches the review turn, then add or run a regression test covering the supplied schema. Done means review either returns schema-conforming output or rejects the flag instead of silently succeeding with prose.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.