openai / openai/codex

measured cost of [false] goal continuations across 3,808 sessions

Open
#44,909 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

agent bug CLI rate-limits
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running?

rust-v0.154.0; rollouts from CLI 0.144–0.154; macOS

What subscription do you have?

ChatGPT Pro 20x

Which model were you using?

No response

What platform is your computer?

No response

What terminal emulator and version are you using (if applicable)?

No response

Codex doctor report

What issue are you seeing?

Summary

With a goal active, Codex restarts the model the moment the thread goes idle, including after turns that made no tool call. The continuation prompt accepts only a live poll as waiting, and the one tool that could wait without spending a turn, clock.sleep, is registered for a single model in the catalog. Waiting for an external process becomes a spin loop where every iteration rereads the full context. On my rollouts, 1.4% of sessions with a goal used 49% of all input tokens. Related reports: #28923, #40929, #24531.

Mechanism

  • extension.rs:175 on_thread_idle calls continue_if_idle, which calls start_turn_if_idle at runtime.rs:454 with no delay. Measured gap between the end of one turn and the next continuation: 0.02–0.05 s, median 0.03 s.
  • token_budget defaults to None (runtime.rs:348).
  • The goal becomes blocked on its own only after three consecutive execution failures (accounting.rs:151). Continuations with zero tool calls do not count toward anything.
  • continuation.md:24 defines a verified wait as polling "a specific process, session, job, or tool handle confirmed live now". The cheapest behaviour that satisfies the prompt is one poll per continuation.
  • spec_plan.rs:1220 registers SleepHandler only when the model catalog lists clock (default SleepToolMode::ModelDriven). In models.json only gpt-6-astra has it. gpt-5.6-sol, gpt-5.5, gpt-5.4 and every custom-provider model get goal mode with no tool that waits without spending a turn.

Measurements

3,808 rollout files, July to September 2026, 59.2B input tokens, 97.8% cache hits.

  • 52 sessions (1.4%) had a goal active and used 29.2B of the 59.2B input tokens. The seven most expensive sessions in the archive all had a goal.
  • One hour of waiting at 160–245K context: 83–188M input tokens in a goal loop without sleep; 10–15M on gpt-6-astra with clock.sleep (60 and 129 calls of 45 s); about 0.5M if the wait were event-driven.
  • Empty continuations: chains of four goal turns with zero tool calls inside 20 seconds, each rereading 65K+ tokens. In one session 14 such turns cost 77M input tokens.
  • Worst session: 173 continuations in 48 minutes, 466 model calls, 149M input tokens, stopped by a 429 from the provider.
  • The weekly limit on a Pro account reached 99% fifteen times in ten weeks, median 23 h after the window opened.

The numbers can be reproduced on any account with codex-rollout-audit, one Python file that reads ~/.codex/sessions locally. Full write-up with charts: https://relux.works/en/blog/codex-goal-token-burn/

Requested changes

All four keep the goal moving; they remove the idle work between moves.

  1. Back off before the next continuation when the previous one made no tool call or only polled, for example 15 s doubling to a cap, instead of restarting at once.
  2. After N consecutive continuations with zero tool calls, move the goal to blocked with a message, the way three execution failures do today.
  3. Register clock.sleep whenever a goal is active regardless of the catalog, or make sleep_tool.mode = "always_on" the default when goals are enabled.
  4. In continuation.md, name clock.sleep and a long empty write_stdin as acceptable forms of a verified wait.
What steps can reproduce the bug?
  1. Codex CLI 0.154, ChatGPT Pro, any model without clock in the catalog, e.g. codex -m gpt-5.6-sol. Keep sleep_tool at its default (ModelDriven), so clock.sleep is not registered.
  2. Start something that finishes in about ten minutes and cannot be observed by event, e.g. in the session: (sleep 600; touch /tmp/done) &.
  3. /goal wait until /tmp/done exists, then print its timestamp and stop.
  4. Leave the session alone for ten minutes.
  5. Observe: a new continuation turn starts every 10–20 s, each one runs a single exec_command (ls /tmp/done, sleep 30, tail on a log) and ends; the next continuation starts within 0.03 s. Some turns contain no tool call at all and are still followed by an immediate restart.
  6. After the goal completes, inspect the rollout: python3 codex_rollout_audit.py session ~/.codex/sessions/<date>/<rollout>.jsonl --commands. The gap_s column shows the 0.02–0.05 s restarts, in_M shows the full context resent on every continuation, and the summary line reports total input tokens spent while waiting for one file to appear.
  7. Repeat with gpt-6-astra: the model calls clock.sleep, continuations are an order of magnitude fewer and the same ten minutes cost 10–15M tokens per hour instead of 83–188M.
What is the expected behavior?

A goal that is waiting for an external process should not cost more than a few model calls per hour. After a continuation with no tool call, or one that only polled, the next continuation should start after a growing delay rather than 0.03 s later, and repeated empty continuations should end in blocked with a message. Any model with an active goal should have clock.sleep available, and the continuation prompt should accept it as a verified wait.

No response

Additional information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with on_thread_idle in codex-rs/ext/goal/src/extension.rs and continue_if_idle/start_turn_if_idle in runtime.rs, then read accounting.rs and continuation.md. Check SleepHandler registration in spec_plan.rs and the model catalog. Done means idle continuations back off, repeated empty continuations become blocked, waiting is available across active goal models, and the prompt recognizes the documented wait forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.