openai / openai/codex

[App] Unapproved max_tokens cap wasted paid API output and discarded all artifacts

Open
#41,490 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

app bug custom-model model-behavior
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of the Codex App are you using (From “About Codex” dialog)?

Unknown (Codex CLI available in the app environment reports 0.150.1)

What subscription do you have?

Unknown / not relevant; third-party OpenRouter usage was billed through a user-provided API key

What platform is your computer?

Darwin 25.5.0 arm64 arm

What issue are you seeing?

While executing a user-requested OpenRouter benchmark, Codex silently added max_tokens: 6000 to every paid generation as a "cost safeguard", without asking the user to approve that experimental or financial tradeoff.

The task required complete self-contained HTML pages. The cap caused paid completions to terminate before the HTML finished. A strict validator then rejected those responses, and the generated content, raw API responses, usage, and costs were not persisted. All 30 jobs were dispatched (3 models × 5 products × 2 runs), but zero HTML artifacts and zero raw responses were retained locally.

This caused avoidable financial loss and made the benchmark unusable. The primary incident cause was the unapproved output-token cap—not output quality or retry behavior.

What steps can reproduce the bug?
  1. Ask Codex App to use a user-supplied OpenRouter key to benchmark several current models by generating complete single-file HTML pages (five products, two runs per model).
  2. Codex creates a Python runner.
  3. Without user instruction or approval, Codex adds "max_tokens": 6000 to every chat-completions request.
  4. Codex starts the entire 30-request matrix with three concurrent workers before running a one-request-per-model preflight.
  5. Model outputs reach the imposed completion limit and end before the HTML document is complete.
  6. The runner validates before persisting and discards each rejected paid response.
  7. The queue continues dispatching after initial truncation/invalidity signals.
  8. Interrupting the thread pool requires multiple interrupts before the process exits.

Relevant generated configuration:

requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    json={
        "model": model,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": 6000,  # added by Codex without user approval
    },
    timeout=240,
)

The exact financial impact could not be reconstructed because per-request usage was only aggregated at full process completion and the run was interrupted.

What is the expected behavior?

Instead of setting defensive "safeguards", codex should use default parameters, and in this case, no max_token should be set. Similar situation happens when codex set a tempurture to reasoning LLM, which is not supported.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is named. Start with the generated Python runner and its requests.post call, then trace how request parameters, validation, persistence, and cancellation are handled. Done means unapproved token or temperature settings are not injected, and rejected responses, usage, costs, and artifacts are retained or handled without silently losing the run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
api, backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.