Extreme cached-context amplification in long-running tool-heavy Codex sessions

Extreme cached-context amplification in long-running tool-heavy Codex sessions

Summary

During long-running autonomous agent workloads (multi-hour sessions with
continuous shell/test/search/git activity and sub-agents), Codex generates an
extreme volume of model requests in which every request carries a very large
cached-input component. Small tool operations (e.g. a single shell command)
are each accounted with roughly 100k–250k input tokens even when the generated
output is only a few hundred tokens. Under this workload a Pro 20x weekly
allowance can be exhausted in roughly one day of continuous use.

This issue reports the amplification mechanism as measured in local Codex
telemetry (rollout JSONL). I am deliberately not asserting that the
server-side rate-limit accounting is incorrect: I cannot determine from local
telemetry how cached input is weighted against the subscription rate limit.
My concern is the client-side behavior that produces this volume, not a claim
about the quota formula.

Related, but distinct, issues I found while searching:

• #44305 — context snowballing measured on short (minutes-long) runs
• #41450 — scheduling-shape A/B study (serial vs. batched independent tool calls)

Those reports document the same underlying mechanism at short time scales.
This report adds large-scale, long-session data (multi-day, 30k+ requests,
85-hour sessions) that, I believe, shows how the same mechanism interacts with
compaction, sub-agents, and the weekly allowance in sustained autonomous use.

Environment

• Codex CLI versions observed in the window: 0.151.0, 0.153.2, 0.153.4, 0.154.0
• OS: Linux (x86_64)
• Plan: Pro 20x (rate-limit plan type reported as pro)
• Models exercised: gpt-6-astra (primary), gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.3-codex-spark
• Context window reported by client: 258,400 tokens

Observed telemetry (31-day window, local rollout JSONL)

Aggregated over 880 session files, 32,149 API requests, 270 sessions
(main + sub-agent threads). All figures are as reported in local telemetry
(token_usage_record per API request). "cached" = cached_input_tokens.

metric	value
API requests	32,149
input tokens	3.998 B
cached input tokens	3.863 B (96.6% of input)
non-cached input	0.135 B (3.4%)
output tokens	18.5 M (0.5%)
reasoning output tokens	6.1 M
total (input+output)	4.017 B
average total per request	~125k
p95 total per request	~215k

Recent 24 hours (timestamp-based):

metric	value
API requests	9,326
input tokens	1.157 B
cached input	97.0% of input
total	1.162 B
average per request	~124.5k

Anomaly distribution across all 32,149 requests:

• 88.7% of requests have cached/input > 90%
• 62.2% of requests have input > 100k while output < 5k
• 185 sessions contain ≥ 10 requests with input ≥ 100k
• 228 sessions exceed 1 M cumulative input tokens

Long-session examples (anonymized)

Top sessions by telemetry-reported total tokens, over the 31-day window:

session	requests	input	total	avg/req	duration
session-A	5,273	714.3 M	717.6 M	~136k	~85.7 h
session-B	4,833	638.3 M	641.5 M	~133k	~85.4 h
session-C	3,711	483.8 M	485.7 M	~131k	~77 h

Top 3 sessions ≈ 1.84 B tokens ≈ 46% of all observed usage in the window.
None of these required manual restart; they were single continuously-running
threads.

Context growth and compaction

Observed per-session pattern (session-A example, first requests):

20.8k → 27.3k → 39.4k → 46.9k → 58.1k → 63.1k → 70.4k → ...
→ 100k+ within ~15 requests (minutes) → band of 100k–250k for thousands of requests

• Context repeatedly reaches 100k–250k; the first request of these sessions
  is ~20k, and 100k is crossed within 5–30 minutes of tool-heavy activity.
• After crossing 100k, context does not decay: the average input of the last
  25% of requests in these sessions is equal to or higher than the first 25%.
• Automatic compaction occurred ~950 times in the window (949 with nearby
  requests available). Median input just before a compaction: ~222k
  (~86% of the 258,400-token window).
• Compaction does work: the immediate next request is still large (median
  ~226k, i.e. no visible drop yet), but 2–3 requests later the input drops to
  ~20–34k. Compaction works, but it does not prevent rapid context
  re-accumulation: under tool-heavy autonomous workloads context rebuilds to
  100k+ again within 5–30 minutes, and the cycle (growth → ~220k → compaction →
  rebuild) repeats many times per session.

So the effective steady state of a long tool-heavy session is: thousands of
model requests each carrying ~100k–250k of (mostly cached) input.

Tool-call amplification

Requests classified by the tool(s) invoked in the same turn:

turn type	requests	avg input	avg output
mixed (several tool kinds)	21,427	~131k	576
shell/exec only	10,599	~110k	574
no tool (final answer etc.)	107	~120k	986

• 10,599 shell-associated requests with average input ~110k tokens and
  average output ~574 tokens. These are the "tiny operation, huge context"
  requests: e.g. a one-line shell command accounted at ~110k input.
• exec-family tool calls: ~118,800 in the window; wait/sleep-type operations:
  ~12,750.
• Tool output sizes: p95 ≈ 35,900 characters; observed maximum ≈ 5.1 M
  characters. A substantial fraction of outputs carries the client's own
  "truncated output" marker, i.e. the client truncates, but p95 outputs of
  ~36k characters still enter context verbatim.

Precise wording on what "input" means here: I do not claim that the same
tokens are literally retransmitted over the network each time. Each model
request is accounted in the local telemetry with a large cached-input
component, and provider-side caching may apply. The amplification is in the
accounted volume: request count × large accounted input.

Model comparison

model	requests	avg total/request	cached%
gpt-6-astra	19,161	~130.5k	97%
gpt-5.6-sol	7,159	~119.5k	97%
gpt-5.6-terra	4,989	~116.2k	96%
gpt-5.6-luna	535	~112.1k	95%
gpt-5.3-codex-spark	305	~70.8k	96%

The gap between the largest and smallest "large" models is ~9–14%
(astra vs sol ≈ 1.09×). Sessions that used only Sol or Terra show the same
100k+/request profile as Astra sessions.

The issue reproduces across models and appears dominated by
context/tool orchestration rather than one specific model.

Client-version regression check

CLI version	requests	avg total/request
0.153.2	2,257	~127.6k
0.153.4	21,797	~126.6k
0.154.0	8,077	~119.7k

I could not identify a clear client-version regression from the available
local telemetry; the newest version in this window is actually ~6% lower
per request. The pattern is present across all recent versions.

Sub-agent contribution

Sub-agent threads (thread_source = subagent) account for roughly 28% of
observed token volume (~1.12 B of 4.0 B). I am not asserting that parent and
child are double-charged for identical tokens; the point is that
parent/child orchestration adds a second, independent tool loop with its own
context growth, so total context processing compounds. Sub-agent threads
running high reasoning effort make this worse in practice.

Rate-limit impact

A detailed re-analysis of the weekly-usage incident found:

• Last snapshot with used_percent=14: 2026-09-09 07:20 UTC.
• First snapshot with used_percent=100: 2026-09-10 11:36 UTC.
• Elapsed time: 28.27 hours.
• Weekly usage increased from 14% used to 100% used, a change of
  86 percentage points.
• Local telemetry during this interval recorded 930.8 M total tokens,
  including 895.1 M cached input tokens (96.6% of input).
• The empirical rate was 10.82 M telemetry tokens per weekly percentage
  point, within the previous calibration range of approximately
  10.17 M–12.44 M tokens per percentage point.
• Usage advanced almost entirely in +1 percentage-point increments, with
  no multi-percentage-point jumps in the recorded snapshots.
• There was no reset_at change, limit_id change, or quota-bucket change.
• Workload throughput during this incident was actually lower than in the
  immediately preceding high-activity control window.

These observations do not support a quota-accounting anomaly in this
incident. Local token telemetry and the weekly usage meter moved broadly
consistently over the interval.

The quota meter appears broadly consistent with the enormous
telemetry-reported token volume. The problem reported here is not an
anomalous quota jump, but why a normal long-running autonomous Codex
workload generates nearly a billion tokens of context processing in
~28 hours, with 96.6% of input reported as cached.

This empirical comparison does not establish the server-side quota formula
or how cached input is weighted against the subscription rate limit.
The concern remains the context-amplification mechanism producing this
volume, rather than a claim that the quota counter is incorrect.

Expected behavior

Long-running autonomous coding is a normal use case for Codex, and I do not
claim it should be free. But:

• Small tool operations should not repeatedly incur unnecessarily large
  effective context processing.
• Stale/large tool outputs should be summarized, truncated, or evicted
  aggressively rather than persisting verbatim in context for hundreds of
  subsequent turns.
• Polling/wait loops should minimize model invocations and context overhead.
• Compaction should prevent runaway repeated context processing, not just
  reset it at 86% of the window.
• Parent/sub-agent orchestration should avoid avoidable duplication of
  context.
• Users should not need to manually restart sessions every 50k–100k tokens
  merely to keep subscription usage practical. (Restarting every ~80k
  "works" as a mitigation, but that is not a satisfactory UX for a product
  whose purpose is long autonomous runs.)

Actual behavior

• Context repeatedly reaches 100k–250k and stays there.
• Thousands of model requests occur at that context size (32k requests in the
  window, of which 62.2% have input > 100k with output < 5k).
• Cached input dominates (~96.6% of all input tokens).
• Small shell-associated requests average ~110k input vs ~574 output.
• Compaction happens (~950 times) but context rebuilds to 100k+ within 5–30
  minutes each time.
• Individual long-running sessions consume hundreds of millions of
  telemetry-reported tokens (top 3 ≈ 1.84 B ≈ 46% of the whole window).
• The most recent 24h exceeded ~1.16 B telemetry-reported tokens.
• As a consequence, the weekly Pro 20x allowance can disappear extremely
  quickly during sustained autonomous use.

Reproduction pattern / observed workload

Not a guaranteed minimal repro — this is the workload shape in which the
effect appears:

1. Start Codex in a medium/large repository.
2. Give it a multi-hour autonomous implementation/refactoring task.
3. Allow normal shell usage, tests, searches, git operations, and sub-agents.
4. Continue the same session rather than manually restarting it.
5. Observe token_usage_record (input / cached_input_tokens) in the rollout
   JSONL per API request.
6. After context reaches > 100k, inspect subsequent tool-heavy turns.
7. Observe many requests where input remains > 100k while generated output is
   very small (< 1k).
8. Observe automatic compaction (~220k) and subsequent rapid re-accumulation
   within minutes.

A simulation of the observed data: if per-request input were capped at 100k,
30% of the window's token volume would not have been accounted; at a 50k
cap, ~61%. The volume is concentrated in a small number of long sessions
(top 3 = 46%).

Possible causes (hypotheses)

Explicitly hypotheses — I am not claiming these are confirmed:

1. Tool outputs remain in context longer than necessary; eviction of stale
   tool results is insufficient for sessions that run for many hours.
2. Repeated polling/wait operations cause additional model turns, each
   carrying the full current context.
3. Parent and sub-agent context processing compounds usage (sub-agents carry
   ~28% of total volume in this window).
4. The automatic compaction threshold (~86% of window) may be too late for
   tool-heavy autonomous workloads: the cost is already paid in the thousands
   of requests that ran at 100k–220k before compaction fires.
5. Tool-output truncation/summarization may be insufficient (p95 ≈ 36k
   characters still enter context).

Suggested improvements

Engineering suggestions, in no particular order:

• Earlier/adaptive compaction for tool-heavy sessions (e.g. trigger below the
  current ~86%-of-window point, or scale the threshold with recent tool
  activity).
• Stronger tool-output truncation/summarization, and eviction of stale tool
  results from context as they age.
• Avoid model round-trips for pure wait/poll operations where the
  architecture permits (batch waits, or wait-without-model for deterministic
  sleep).
• Context eviction based on relevance rather than strict LRU.
• Incremental/delta context mechanisms where architecture permits (present
  only changed state after edits/tool results instead of the full transcript).
• More efficient parent/sub-agent context handoff (avoid re-presenting the
  parent's full context to a child that only needs a task brief).
• Expose per-turn effective quota cost in the UI/CLI.
• Expose the cached vs uncached contribution to rate-limit consumption, so
  users can see what drives their allowance.
• A warning when a session enters pathological context amplification
  (e.g. N consecutive requests with input > 100k and output < 1k).
• Optional automatic session checkpoint/handoff so users are not forced to
  manually restart sessions to keep subscription usage practical.

Additional notes

• All figures are from local Codex telemetry on a single Pro 20x account over
  31 days; session and thread identifiers are anonymized and no source code,
  prompts, tool outputs, or log excerpts are included here.
• The same per-request amplification shows up on every model tested
  (astra/sol/terra/luna) and on every CLI version in the window, so it is not
  attributable to a single model or a recent regression.
• I can provide additional aggregated telemetry or a sanitized analysis
  script if useful.

Begin with the reproduction pattern and inspect rollout JSONL records, especially token_usage_record, cached_input_tokens, context growth, compaction, and tool-heavy turns. Compare long-running sessions before and after compaction, including parent and sub-agent threads; done means reducing unnecessary context amplification while preserving autonomous tool use and verifying the observed request and token patterns improve.