[Code Mode] Reproducible token amplification: one AGENTS.md rule cut cumulative input by 81.6%
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
[Code Mode] Reproducible token amplification: one AGENTS.md rule cut cumulative input by 81.6%
Read this first: immediate zero-patch mitigation
If Codex is performing many independent, read-only checks, add the following to
the project AGENTS.md, or use it as the first instruction in a new session:
In Code Mode, when the current-stage checks are already known to be independent,
read-only, and conflict-free, put them in one exec and run them concurrently.
Use Promise.allSettled when useful partial results should be preserved and inspect
every result; use Promise.all only when any failure must abort the batch. Keep
dependent, writing, approval-sensitive, adaptive, and waiting operations serial.
Do not poll a running process every second; use a wait window appropriate to its
expected duration. Do not repeat completed checks. These rules must not reduce
task scope, reasoning depth, verification, tool coverage, or answer quality.
This is a scheduling instruction, not a model downgrade or a task shortcut.
In my controlled A/B test, it reduced cumulative logical input by 81.606%
while preserving 100% task coverage:
| Control: serial tool plan | Treatment: scheduling guidance | |
|---|---|---|
| outer model cycles | 60 | 15 |
| cumulative logical input | 11,100,973 | 2,041,915 |
| cumulative cached input | 9,980,996 | 1,827,703 |
| required checks | 50/50 | 50/50 |
| nested tool calls | 86 | 86 |
| errors | 0 | 0 |
That is 9,059,058 fewer logical input tokens, approximately 82% less
for this dense high-context workload. The same task was completed, the same
nested calls were made, and no model, tool, reasoning, context, compaction, or
verification capability was reduced.
The complete investigation, including the local-history analysis, deterministic
reproduction, wait/retry/fallback tests, context sweep, limitations, and source
reference is in this issue. Earlier related Code Mode evidence is in
#35050; the related large-input
quota symptom is in #41369.
What is the bug/failure mode?
The problem is not that GPT-5.6 Sol is abnormal on every turn.
The measurable failure mode is:
cumulative logical input ≈ Σ input presented to each outer model cycle
≈ carried context × outer model cycle count
For already-known independent checks, an inefficient plan can repeatedly return
to the outer model after one small tool call:
model → tool A → model → tool B → model → tool C
The growing transcript is then logically presented again and again. A high
cached-input ratio is expected in this pattern; it indicates repeated,
cacheable history, not by itself a cache or wire-transport defect.
The lower-cost equivalent for known-independent work is:
model → one exec(Promise.all/Promise.allSettled(A, B, C)) → model
This does not apply to dependent, dynamic, writing, side-effectful,
approval-gated, or waiting operations. Those should remain serial.
The issue is therefore a client/model-interaction scheduling failure that can
amplify quota usage. It may explain a substantial part of the perceived
“quota shrink” symptom in dense tool workloads, but it is not a claim that every
quota report has the same cause or that every billing issue is explained here.
Why can the same Sol look normal sometimes?
A streamed scan of recent local session metadata found:
- 60 session files;
- 205 identifiable Sol-labelled turns;
- 144 main turns and 61 child turns; and
- normal, moderate, and extreme outliers.
The observable difference was the outer-cycle shape, not merely the model name.
Normal turns tend to batch independent calls. Abnormal turns tend to split them
into singleton calls and re-enter the model repeatedly while context grows.
Examples from same-session state changes were:
| turn shape | input delta | outer cycles | exec / nested calls |
waits | compactions |
|---|---|---|---|---|---|
| short/normal | 21,667 | 1 | 0 / 0 | 0 | 0 |
| abnormal | 4,528,553 | 51 | 33 / 35 | 5 | 0 |
| extreme | 36,040,574 | 303 | 245 / 272 | 49 | 2 |
| later shorter turn | 621,343 | 3 | 1 / 3 | 0 | 0 |
Across 110 anchored main Sol turns, the strongest predictor was average
per-cycle input multiplied by outer cycles:
- Pearson correlation:
0.9905; - Spearman correlation:
0.9841.
The corrected linear fit was:
inputDelta = -34,031 + 0.9230 × (average_last_input × outer_cycles)
R² = 0.9811
Singleton-exec ratio alone was not a sufficient historical classifier because
the metadata cannot prove whether every singleton was semantically independent.
The deterministic fixture below isolates the causal cost of serializing work
that is known to be independent.
Reproduction with the unmodified official client
The primary A/B used the installed, unmodified official Codex binary:
codex-cli 0.150.0-alpha.12.2;- binary SHA256:
34E9CFE7D5BBCEC306FE6AB3FD502A713A7A1F0FB644C11AD2990FC80599FD4F; - temporary isolated
CODEX_HOME; - deterministic localhost fake Responses server;
- 180K synthetic context;
- 50 independent read-only checks; and
- the same tool workload and coverage requirements in both arms.
The client was configured for GPT-5.6 Sol. The fake server replaced only the
upstream at the network boundary, recorded usage/tool/completion events, and
never called a real model, production OpenAI endpoint, or production Sub2API.
Control and treatment used the same official client path. The only experimental
variable was the scheduling guidance that biases known-independent calls toward
one programmatic batch.
The treatment was repeated:
| run | outer cycles | logical input | cached input | exec / nested |
coverage |
|---|---|---|---|---|---|
| control | 60 | 11,100,973 | 9,980,996 | 59 / 86 | 50/50 |
| treatment 1 | 15 | 2,041,915 | 1,827,703 | 14 / 86 | 50/50 |
| treatment 2 | 15 | 2,041,940 | 1,827,727 | 14 / 86 | 50/50 |
Both treatment runs had zero errors and complete coverage. A mixed-command
fixture was also repeated: both arms executed the same command shape
(Get-Content×49, Select-String×7, Get-Item×6) and covered all 20 checks;
only the outer scheduling shape changed.
This fake-server test is intentionally not described as reproducing Sol's
learned sampling policy. It tests the real official client path and makes the
cost of the two scheduling shapes deterministic. The local session history
supplies the real normal/abnormal variation; the isolated fixture supplies
repeatable causal accounting.
Context-size sweep
A separate deterministic harness sweep used the same 20 independent checks at
different synthetic context sizes. Its logical usage is approximately JSON
characters divided by four, so these values demonstrate structural amplification
rather than production tokenizer or billing values.
| context | batched input | serial input | reduction | coverage |
|---|---|---|---|---|
| 20K | 97,999 | 735,875 | 86.7% | 20/20 both |
| 100K | 491,240 | 2,693,946 | 81.8% | 20/20 both |
| 180K | 1,235,604 | 5,004,702 | 75.3% | 20/20 both |
| 220K | 1,731,486 | 6,283,941 | 72.4% | 20/20 both |
At 180K with 50 checks, the serial plan reached 11,099,443 input versus
2,038,803 for the batched plan, with 50/50 coverage in both arms.
The headline range is therefore a measured high-context amplification range,
not a universal promise for every task.
Accounting for the originally quoted 11.16M turn
One reproducibility correction is important: the local turn previously described
as an 11.16M GPT-5.6 Sol turn is identified by its recorded metadata as
gpt-5.6-luna, not Sol.
That turn contained:
- 95 usage cycles;
+11,156,761input;+10,507,264cached input;- 86
execinvocations; - 87 nested calls;
- 85 singleton exec cells and one two-call exec cell;
- wait polling: 0;
- retry/fallback/incomplete markers: 0; and
- approximate retained tool output of only 3.439% of the input delta.
The nearby correctly identified Sol turn had 50 cycles and +4,416,127 input,
with 38 singleton nested-call execs, wait 0, and no retry/fallback/hidden-model
signal.
The numerical phenomenon is real, but the original model label must not be
repeated. The available evidence does not establish that Sol is intrinsically
more susceptible than Terra or Luna.
The JSONL schema does not provide mutually exclusive labels for “normal tool
continuation” and “inefficient serialization”. I therefore do not invent an
exact semantic partition. The logs prove that the dominant identifiable shape
was 85 singleton exec cells plus one two-call exec, while wait, retry, fallback,
and hidden-agent signals were absent.
Other hypotheses tested
Wait polling
A deterministic approximately-30-second process was run with two wait policies:
| policy | cycles | logical/cached input | short waits |
|---|---|---|---|
| one-second polling | 7 | 79,550 / 61,753 | 6 |
| reasonable wait window | 2 | 22,046 / 9,997 | 0 |
One-second polling is a real secondary amplifier:
wait → model → wait → model
It cannot explain the largest target ledgers by itself, and the 11.16M target
turn had no wait events.
Retry and fallback
One pre-output connection failure added approximately one cycle in the fake
retry test. A fake WebSocket failure followed by retry and HTTP fallback
completed with the same coverage and essentially the HTTP-baseline logical
input. These mechanisms can amplify usage, but they are not the primary cause
of the measured serial-tool explosion.
Tool output
Retained tool output was approximately:
- 10.13% of the input delta in the nearby real Sol turn;
- 3.439% of the input delta in the 11.16M Luna turn.
Repeated context, not newly produced tool output, was the main contributor.
Provider and hidden agents
The local target provider metadata was openai, and the official client
reproduced the structure against a local fake Responses endpoint. Sub2API is
therefore not required to reproduce this particular behavior.
No hidden Luna/Terra/guardian/auto-review/subagent signal was present in the
target attribution ledger. A fair controlled Sol-versus-Terra-versus-Luna
susceptibility comparison has not been established.
Why the mitigation preserves capability
The A/B test kept the following unchanged:
- official binary;
- model and model configuration;
- context;
- tool workload;
- 50 independent checks;
- 86 nested calls;
- verification requirements;
- final coverage; and
- error handling.
The instruction only changes this representation:
model → tool A → model → tool B → model → tool C
into this representation when the calls are already known to be independent:
model → one exec(Promise.all/Promise.allSettled(A, B, C)) → model
All results must still be preserved and inspected. Promise.allSettled is used
when partial failures should not cancel useful results. Promise.all is used
only when any failure must abort the batch.
Dependent, state-changing, writing, approval-sensitive, adaptive, and waiting
operations remain serial.
This is scheduling efficiency, not reduced reasoning, reduced verification, or
reduced tool capability.
Candidate upstream reference change
A minimal reviewable source-level reference is available here:
https://github.com/larryllr/codex/commit/23dfc40c5af9b2ede0ff7426beaf5a392684cb2a
It adds safe scheduling boundaries to the Code Mode exec description and adds
a regression test. It does not modify the runtime binary used for the A/B test.
No Responses serializer, model metadata, context setting, compaction setting,
reasoning setting, tool implementation, retry logic, transport, or binary is
changed.
This fork commit is a maintainer-facing reference, not a request for users to
install a patched binary or maintain a private Codex fork. The practical
zero-patch mitigation is the supported first-message or AGENTS.md instruction.
If upstream later fixes the default scheduler, the workaround should be removed
after verification, and this fork reference should no longer be needed.
Proposed acceptance criteria for an upstream fix
An upstream implementation should be tested against the same fixture and should:
- reduce outer cycles and cumulative logical input in the serial failure case;
- preserve 100% of required checks, nested calls, error handling, and final
findings; - leave context, compaction, reasoning, model, tool, retry, and transport
capabilities unchanged; and - make the AGENTS.md workaround removable after an official release is verified.
Summary
GPT-5.6 Sol is not abnormal on every turn.
Dense tool workflows can sometimes enter a high-outer-cycle scheduling state.
When a large context is replayed across many unnecessary model cycles, cumulative
logical input grows into the millions.
The controlled result was:
BEFORE: 60 cycles, 11,100,973 input, 9,980,996 cached
AFTER: 15 cycles, 2,041,915 input, 1,827,703 cached
Both arms completed the same required work with 100% coverage and zero errors.
This is a reproducible Code Mode token-amplification failure mode that may
explain a significant portion of the perceived “quota shrink” effect in dense
tool workloads. It is actionable today through an AGENTS.md scheduling rule,
while the source-level reference provides a concrete upstream regression test.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked candidate source commit and inspect the Code Mode exec description and its regression test. Verify whether safe batching boundaries reduce repeated outer cycles while preserving serial handling for dependent, writing, approval-sensitive, adaptive, and waiting operations; done means the regression coverage confirms the required checks and task scope remain intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100