openai / openai/codex-plugin-cc

review --json emits no `result` (and no `parseError`): the built-in reviewer path never produces schema-shaped output, unlike adversarial-review

Open
#679 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
33.3k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

Summary

In 1.0.6, codex-companion.mjs review --json returns a payload with no result key at all — and no rawOutput or parseError either, so a consumer cannot distinguish "structured output was not produced" from "structured output failed to parse". adversarial-review --json on the same repository, same working tree, returns result conforming to the plugin's own schemas/review-output.schema.json.

The two subcommands therefore emit two different --json shapes, and only one of them is machine-readable. Nothing in the payload marks which shape a consumer received.

I hit this building an automated benchmark that scores the plugin's reviewer against planted ground truth: the harness reads payload.result, and every case skipped with no result in payload while the companion itself exited 0 and looked healthy.

Repro

Minimal git repo, one file, one breaking change in the working tree (the change deletes module.exports):

node scripts/codex-companion.mjs review --scope working-tree --json --cwd /path/to/repo

Top-level keys of the emitted JSON:

review, target, threadId, sourceThreadId, codex

codex.stdout holds prose:

The change removes the module's only export, breaking all existing consumers that import the function.
Review comment:

  • [P1] Restore the module export — …

Same repo, same working tree, other subcommand:

node scripts/codex-companion.mjs adversarial-review --scope working-tree --json --cwd /path/to/repo

Top-level keys:

review, target, threadId, context, codex, result, rawOutput, parseError, reasoningSummary

with parseError: null and

{"verdict":"needs-attention","summary":"Do not ship: the change removes the module's public export…","findings":[{"severity":"high","title":"Function is no longer exported", …}]}

Why the shapes differ

executeReviewRun branches on the review name (scripts/codex-companion.mjs, ~L357–L455):

  • reviewName === "Review"runAppServerReview(...), i.e. codex's built-in reviewer. No outputSchema is passed. The payload is assembled as { review, target, threadId, sourceThreadId, codex: { status, stderr, stdout: result.reviewText, reasoning } } — there is no result field in that object literal.
  • any other review name (e.g. Adversarial Review) → the prompt-template path, which passes outputSchema: readOutputSchema(REVIEW_SCHEMA) and runs the response through parseStructuredOutput, producing result / rawOutput / parseError.

So the absence is by construction, not a parse failure. parseStructuredOutput itself always returns parsed and parseError (null on failure) — it is simply never called on the native path.

Impact

  • schemas/review-output.schema.json ships with the plugin but the default review never satisfies it.
  • A --json consumer must special-case the review name to know whether findings are readable, and there is no field in the payload that says which path produced it.
  • The failure is silent in the direction that matters: no result, and no parseError explaining its absence.

This looks distinct from #496, which is about the schema path failing to conform on long multi-tool-call turns; here the schema path is not taken at all. #496 also describes review as sharing the runAppServerTurn path, which no longer matches 1.0.6.

Suggestions (any one would resolve it for a consumer)

  1. Map the built-in reviewer's findings into the same review-output.schema.json shape and emit them as result on the review path.
  2. Failing that, emit an explicit result: null plus a parseError-style reason (e.g. "built-in reviewer returns prose; use adversarial-review for structured output") so the absence is stated rather than inferred.
  3. And/or document the two --json shapes, keyed by subcommand, so consumers can branch deliberately.

Environment

  • codex plugin 1.0.6 (installed via the Claude Code marketplace)
  • codex-cli 0.149.0
  • Node v24.14.1
  • Windows 11 (10.0.26200)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce both commands with --json, then inspect executeReviewRun in scripts/codex-companion.mjs around lines 357–455, including runAppServerReview and parseStructuredOutput. Compare the built-in review payload with schemas/review-output.schema.json and the adversarial-review fields; done means the review JSON contract explicitly accounts for structured output availability rather than silently omitting it.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.