[App] Unapproved max_tokens cap wasted paid API output and discarded all artifacts
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What version of the Codex App are you using (From “About Codex” dialog)?
Unknown (Codex CLI available in the app environment reports 0.150.1)
What subscription do you have?
Unknown / not relevant; third-party OpenRouter usage was billed through a user-provided API key
What platform is your computer?
Darwin 25.5.0 arm64 arm
What issue are you seeing?
While executing a user-requested OpenRouter benchmark, Codex silently added max_tokens: 6000 to every paid generation as a "cost safeguard", without asking the user to approve that experimental or financial tradeoff.
The task required complete self-contained HTML pages. The cap caused paid completions to terminate before the HTML finished. A strict validator then rejected those responses, and the generated content, raw API responses, usage, and costs were not persisted. All 30 jobs were dispatched (3 models × 5 products × 2 runs), but zero HTML artifacts and zero raw responses were retained locally.
This caused avoidable financial loss and made the benchmark unusable. The primary incident cause was the unapproved output-token cap—not output quality or retry behavior.
What steps can reproduce the bug?
- Ask Codex App to use a user-supplied OpenRouter key to benchmark several current models by generating complete single-file HTML pages (five products, two runs per model).
- Codex creates a Python runner.
- Without user instruction or approval, Codex adds
"max_tokens": 6000to every chat-completions request. - Codex starts the entire 30-request matrix with three concurrent workers before running a one-request-per-model preflight.
- Model outputs reach the imposed completion limit and end before the HTML document is complete.
- The runner validates before persisting and discards each rejected paid response.
- The queue continues dispatching after initial truncation/invalidity signals.
- Interrupting the thread pool requires multiple interrupts before the process exits.
Relevant generated configuration:
requests.post(
"https://openrouter.ai/api/v1/chat/completions",
json={
"model": model,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": 6000, # added by Codex without user approval
},
timeout=240,
)
The exact financial impact could not be reconstructed because per-request usage was only aggregated at full process completion and the run was interrupted.
What is the expected behavior?
Instead of setting defensive "safeguards", codex should use default parameters, and in this case, no max_token should be set. Similar situation happens when codex set a tempurture to reasoning LLM, which is not supported.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file or test is named. Start with the generated Python runner and its requests.post call, then trace how request parameters, validation, persistence, and cancellation are handled. Done means unapproved token or temperature settings are not injected, and rejected responses, usage, costs, and artifacts are retained or handled without silently losing the run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- api, backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100