Codex subagents drain full week quota overnight - usage counting broken

Open
#35,463 15 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
backend, cli

Research direction

Start with codex-rs/core/src/config/mod.rs and compare the spawn entry points in codex-rs/core/src/tools/handlers/multi_agents/spawn.rs and multi_agents_v2/spawn.rs, then inspect codex-rs/core/src/client.rs for fork connection setup. Reproduce a nested MultiAgent V2 fork and compare parent and child rollout JSONL token_count events. Done means the reported depth, bookkeeping replay, and quota-accounting behaviors are addressed and verified.

Written by the indexing model from the issue text.

Description

bug CLI rate-limits subagent
What version of Codex CLI is running?

0.145.0

What subscription do you have?

Pro 20x

Which model were you using?

gpt-5.6-sol

What platform is your computer?

MacOS 26.3.1 (Darwin 25.3.0, arm64)

What terminal emulator and version are you using (if applicable)?

No response

Codex doctor report

What issue are you seeing?

My entire Pro plan usage quota was drained to 0% overnight on 2026-07-26, correlated specifically with heavy use of the MultiAgent V2 subagent feature (multi_agent_version: "v2"). This does not happen on days when I don't use subagents.

This happened across two independent projects/workspaces on the same machine that day. In both cases I instructed the top-level orchestrator agent to delegate work to subagents for actual task execution. I was using reasoning_effort: xhigh, not any higher/"ultra"-tier workflow preset. Despite that, both workspaces independently produced fork trees up to 4 levels deep (a subagent's subagent's subagent's subagent), which was not something I asked for. As detailed in problem N3 below, this turns out not to be a configuration oversight on my part — there is currently no way to bound nesting depth under MultiAgent V2 at all.

I audited my local ~/.codex/sessions rollout logs for that day in detail and found three distinct, compounding problems:

1. Every subagent fork replays its entire ancestor lineage's bookkeeping into its own log file, verbatim

When a subagent forks, its rollout JSONL is seeded with a burst of thousands of token_count events — a byte-for-byte copy of everything every ancestor in its fork chain had already logged — all written within 1-2 seconds and re-timestamped to the fork's own creation time.

Concretely, on 2026-07-26 I had 129 sessions (root thread originally created 2026-07-17, recursively forking subagents up to 4 levels deep). Representative examples from a single fork chain (root → A → B, C, D...):

  • Parent session: 23,743 total logged token_count events, of which 22,919 (96.5%) were themselves an inherited burst from its parent, leaving only 824 genuinely new events across its real ~4.7 hour working span.
  • Two children forked from that same parent: 23,135 total events (23,108 replayed / 27 real) and 23,468 total events (23,388 replayed / 80 real) respectively.

I compared the replayed burst in a child session against the parent's own historical records index-by-index: 100% of the (input_tokens, cached_input_tokens, output_tokens) triplets matched exactly, in the same order — the only difference was the timestamp. This is unambiguously a local copy/replay of already-recorded bookkeeping, not new model calls (23,000+ in ~2 seconds is not a plausible real request rate).

This inflates every downstream reporting tool that sums token_count events (e.g. ccusage — see ccusage/ccusage#950) by up to ~90x. My own reconstruction of "real" (non-replayed) usage for the day came to roughly 1.5% of what naive summation reports.

I want to be explicit that I cannot confirm from local logs alone whether this replayed data is inert with respect to my actual billed/metered usage, or whether it's also the direct cause of the real quota drain described below. If whatever system computes my account's real usage quota draws on this same per-session token telemetry — rather than exclusively on independently-verified real API request records — then this bug would not just be inflating third-party tools like ccusage, it would be inflating my actual billed usage too, once per fork in the tree. I'd like this checked directly rather than assumed away.

2. The real backend usage quota was actually drained

Regardless of how problem N1 is ultimately explained, my account's real weekly usage quota was fully consumed overnight. This is a genuine backend-side accounting problem tied to subagent spawning, not something I can attribute to a local display/logging bug alone.

From reading codex-rs/core/src/client.rs: a fresh subagent fork has no established websocket connection and no previous_response_id to chain from (prepare_websocket_request returns (None, false) when get_last_response() is empty), so every fork must open a brand-new connection and send a non-incremental first request. With a deep/wide fork tree (dozens of forks within a single hour, several clustered within the same minute), this produces a burst of new-connection establishment that looks structurally identical to the pattern that originally tripped the anti-abuse/rate-limiter bug in openai/codex#9748 (which was reported fixed server-side in February 2026, but scoped to MultiAgent V1's concurrent-spawn trigger — I could not find confirmation that the fix covers V2's tree-forking spawn pattern).

This may also be connected to the broader, still-open quota-accounting regression described in openai/codex#31668, where OpenAI's own team acknowledged anti-abuse/fraud-prevention systems overflagging ordinary usage — though that issue includes reports with no subagents involved at all, so it may be a distinct or overlapping cause rather than the same one.

3. MultiAgent V2 has no enforced nesting-depth limit at all

There is an existing config field intended to cap subagent nesting depth, agents.max_depth (agent_max_depth internally, in codex-rs/core/src/config/mod.rs). Its own doc comment says exactly why it didn't help me:

/// Maximum nesting depth for V1 agent threads. Ignored by V2.
pub agent_max_depth: i32,

I confirmed this is accurate — not just a stale comment — by reading both spawn handlers directly:

  • codex-rs/core/src/tools/handlers/multi_agents/spawn.rs (V1): computes child_depth via next_thread_spawn_depth(&session_source), reads turn.config.agent_max_depth, and calls exceeds_thread_spawn_depth_limit(child_depth, max_depth) — rejecting the spawn with "Agent depth limit reached. Solve the task yourself." if it's exceeded.
  • codex-rs/core/src/tools/handlers/multi_agents_v2/spawn.rs (V2 — not something I opted into; see note below): computes child_depth via the exact same next_thread_spawn_depth call (line 60), but never calls exceeds_thread_spawn_depth_limit, or any other depth check, anywhere in the file.

Worth noting this isn't a niche configuration choice on my part: openai/codex#31097 documents that GPT-5.5 forces MultiAgentV2 even when it's explicitly disabled in config.toml and via command-line overrides. I was on gpt-5.6-sol, the same model line, and never selected V2 myself — so whatever is broken here isn't limited to users who deliberately opted into V2, it applies to anyone using this model family with subagents at all.

So there is currently no configuration that bounds fork-tree depth under MultiAgent V2 — not "the default is too permissive," but the enforcement path that exists for V1 has no V2 equivalent. Combined with problems N1 and N2 above, this is close to a worst case: unlimited recursion depth, where every level replays its full ancestor history into its own log (N1) and every level opens a new, non-incremental, potentially quota-draining connection (N2).

What steps can reproduce the bug?
  1. Start a long-lived Codex session on gpt-5.6-sol with reasoning_effort: xhigh (MultiAgent V2 does not need to be deliberately selected — per openai/codex#31097, this model line forces V2 regardless of config). Note that agents.max_depth has no effect under V2 regardless of whether it's set (see problem N3), and fork_turns defaults to "all" / full-history fork if left unset (per openai/codex#34061).
  2. Instruct the orchestrator agent to delegate pieces of work to subagents (a single level of delegation, as normally expected). Do not request or configure recursive/nested spawning.
  3. Observe that subagents themselves spawn further subagents unprompted, several levels deep, with no depth limit enforced and full ancestor history replayed at every level.
  4. Let this run for several hours, accumulating dozens of forks across the resulting tree, several minutes apart or clustered within the same minute.
  5. Observe: (a) local rollout logs balloon with replayed bookkeeping (confirmable by diffing a child's early token_count sequence against its parent's), and (b) the account's real weekly usage quota drains to 100% far faster than the actual new work performed would justify.
What is the expected behavior?
  • Forking a subagent should not replay/duplicate the parent's entire historical token_count/bookkeeping records into the child's own log — at minimum this should be a reference to the parent session rather than a duplicated copy, so downstream tools don't double/N-count it.
  • Real backend usage quota consumption should scale with actual new inference performed, not with the number and depth of forks in a session tree. Establishing a new fork's connection is a one-time, filtered-history event (per the #34061 investigation) and shouldn't be able to single-handedly exhaust a subscription's full usage window.
  • Subagents should not spawn further nested subagents unless that is explicitly configured/requested. A single requested level of delegation should not silently become a multi-level recursive fork tree. Concretely: agents.max_depth should be enforced by MultiAgent V2's spawn handler the same way it already is for V1, not silently ignored.
  • If a rate-limiter/anti-abuse system is what's actually draining the quota (as in #9748 and #31668), it should not be triggered by ordinary, sequential subagent spawning under MultiAgent V2, and should never zero out the visible/billed usage quota as a side effect of being tripped.
Additional information
  • Related issues that this report ties together: #9748, #22779, #31097, #34061, #33447, #31668, ccusage/ccusage#950.
  • I have local rollout JSONL evidence (fork chains, byte-for-byte replay comparison, timing analysis) available if useful for debugging — happy to share sanitized excerpts privately rather than post full logs here.
Dominant language
Rust
Stars
125k
Forks
19.5k
Avg merge
1m
Merged PRs (30d)
1k

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openai/codex

All issues in openai/codex

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.