openai / openai/codex

Post-Aug-29 abnormal usage: model-cycle/context-replay amplification may remain after recent efficiency fixes

Open
#41,605 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What issue are you seeing?

This report concerns abnormal usage observed after the Aug 29 Codex usage-efficiency announcement and paid-user usage reset.

In a fresh post-reset usage window, weekly usage still depleted by more than 80 percentage points over roughly four hours of ordinary repository-development activity.

For the purpose of this report, I am treating the previously announced waste mechanisms as fixed and asking whether a technically distinct amplification path may remain.

This report does not use that post-reset meter observation to identify the mechanism. It establishes only that a severe user-visible symptom was observed after the Aug 29 announcement/reset boundary.

The mechanism analysis below instead relies on:

  1. public source changes corresponding to recent usage-efficiency fixes;
  2. real-session telemetry already reported in #41541; and
  3. controlled structural reproduction in #41450.

The public evidence points to a distinct possible amplification mechanism:

large retained context × high outer model-cycle frequency

Recent real-session telemetry reports substantially higher model-step frequency in the measured 0.150.x cohort, while a controlled reproduction using the official 0.150.0-alpha.12.2 client demonstrates that additional outer model re-entry can multiply cumulative logical input by several times even when the underlying tool workload and required task coverage remain unchanged.

The recent publicly visible usage-efficiency fixes appear to address different failure modes. I could not identify a corresponding public change that directly eliminates unnecessary outer model re-entry over large retained contexts.


1. Recent usage-efficiency fixes address specific waste classes

Several recent public changes are clearly directed toward reducing avoidable model work.

Retained-image budgeting

Commit:

528fd7ace5ec0a1c2a387dcb9c76a09f3fa011ee

PR:

#40994 — Enable retained-image budgeting by default

This change enables retained-image budgeting by default and allows older retained images to be removed when the remote-compaction budget is exceeded.

Its visible direction is to prevent historical images from unnecessarily remaining in model-visible context and contributing repeatedly to compaction pressure.

This is usage-reducing.


Memory consolidation stop-hook isolation

Commit:

7c6eb0eef113ddc16ae5b207ac9add364b489798

PR:

#40587 — Scope stop hooks for memory consolidation

This change prevents background memory-consolidation work from inheriting user/project/session/plugin stop hooks that can interfere with normal worker completion.

It also avoids repeatedly feeding managed rejection states back into an unattended worker loop.

Again, the visible direction is to reduce unintended background work.


Repeated execution-host failure guard

Commit:

62b458c93151595cdf2b5ef5e37aa3d8b5613aeb

PR:

#41454 — Block goals after repeated execution host failures

This introduces a bounded failure streak for qualifying execution-host failures and blocks the goal after repeated failed attempts rather than allowing the same failure class to continue indefinitely.

Successful tool execution or goal replacement resets the streak.

This is also clearly usage-reducing.


These changes appear technically consistent with their intended purpose.

The concern here is not that these fixes are incorrect.

The concern is that they address failure modes structurally different from the outer-model-cycle/context-replay amplification described below.


2. Real-session telemetry shows higher model-step density in the recent runtime period

Issue #41541 compares five earlier real development sessions using 0.147/0.148 with five recent sessions using 0.150.x.

The selected sessions use:

  • source = vscode
  • plan_type = prolite
  • reasoning_effort = high
  • no Ultra mode
  • no manually launched subagents

The comparison also excludes substantial idle periods, quota resets occurring inside the measured interval, and overlapping main VS Code sessions.

The reported cohort-level results are:

Metric Stable 0.147/0.148 Recent 0.150.x Change
Token throughput 0.499M/min 0.662M/min +32.6%
Model-step frequency 3.515/min 4.610/min +31.1%
Approx. tokens/model step ~142K ~144K ~+1%
Cached-input ratio 98.076% 97.731% −0.35 pp

The important observation is that average token volume per model step is almost unchanged:

~142K → ~144K

while model-step frequency increases substantially:

3.515 → 4.610 steps/min

The increase in token throughput closely tracks the increase in model-step frequency.

This is consistent with the measured recent-runtime cohort performing more model/tool continuation cycles per unit of wall-clock time rather than simply issuing dramatically larger individual model requests.

This does not establish that a specific 0.150.x client commit caused the difference.

The dataset establishes an observed difference between the compared runtime periods. Client code, model behavior, server/runtime behavior, and other contemporaneous changes are not fully separated by that comparison.


3. Controlled reproduction shows the cost of additional outer model cycles

Issue #41450 isolates the structural cost of repeated outer model re-entry.

The primary controlled A/B uses:

  • official, unmodified codex-cli 0.150.0-alpha.12.2;
  • an isolated temporary CODEX_HOME;
  • a deterministic fake Responses server;
  • 180K synthetic context;
  • 50 independent required checks;
  • the same nested tool workload and task-coverage requirements.

The reported result is:

Metric Serial scheduling Batched scheduling
Outer model cycles 60 15
Cumulative logical input 11,100,973 2,041,915
Cached input 9,980,996 1,827,703
Nested tool calls 86 86
Required checks 50/50 50/50
Errors 0 0

The underlying required work and coverage are preserved.

The major execution-shape difference is approximately:

model
→ tool A
→ model
→ tool B
→ model
→ tool C

versus:

model
→ batch(A, B, C)
→ model

In that specific controlled workload:

11.10M / 2.04M ≈ 5.4x

more cumulative logical input is required by the serial execution shape.

The 5.4x figure is specific to that fixture and should not be generalized as a universal Codex multiplier.

The more general structural result is approximately:

cumulative logical input
≈
context presented per outer model cycle
×
number of outer model cycles

When retained context is large, unnecessary outer model re-entry can therefore become very expensive.

The fake-server fixture does not reproduce a production model's learned scheduling policy. Its value here is narrower: it deterministically isolates the cost difference between the two execution shapes through the official client path.


4. The two observations make context-replay amplification a plausible contributor

#41541 and #41450 provide different kinds of evidence.

#41541 provides real-session observational evidence that:

model steps / minute increased

in its measured recent-runtime cohort.

#41450 provides controlled structural evidence that:

more outer model cycles
×
large retained context

can strongly increase cumulative logical input even when the required tool workload is held constant.

Together, these observations make the following mechanism a plausible contributor:

higher outer model-cycle density
        ↓
more frequent replay of retained context
        ↓
higher cumulative logical input
        ↓
more model work for the same class of tool-heavy workflow

They do not establish that the additional model steps observed in #41541 are specifically the same unnecessary serial re-entry isolated by #41450.

That connection remains a hypothesis rather than a directly measured causal link.

The important point is that the mechanism is technically distinct from fixes for:

  • retained-image growth;
  • runaway memory workers;
  • repeated execution-host failures.

Those fixes can function correctly while outer-cycle/context-replay amplification remains possible.


5. Why the recent fixes do not obviously cover this mechanism

The recent publicly visible fixes primarily reduce avoidable work by removing or terminating specific problematic states.

For example:

too many retained images
→ trim retained images
background worker prevented from finishing
→ isolate stop hooks / terminate correctly
repeated execution-host failure
→ stop after bounded failure streak

The outer-cycle/context-replay mechanism has a different shape:

large retained context
        ↓
model
        ↓
small tool operation
        ↓
large retained context replayed
        ↓
model
        ↓
another small tool operation
        ↓
large retained context replayed
        ↓
...

No individual operation in this sequence necessarily represents a runaway worker, retry loop, or malformed retained object.

The amplification arises from the execution graph itself: repeatedly returning to the outer model between operations that may already be known to be independent.

That makes this closer to a scheduling inefficiency than to the failure modes addressed by the fixes above.


6. Earlier batching evidence points in the same direction, with an important caveat

Issue #35050 contains earlier controlled batching experiments that point in the same general direction: reducing outer model/tool round trips was associated with substantial reductions in model cycles and cached-input processing.

However, its original weighted-usage headline requires an important caveat.

The author later reported that some batched exec responses exceeded Code Mode's shared output limit and were truncated. Because of this, the original 27–45% weighted-usage savings range was explicitly marked provisional.

A subsequent truncation audit restored explicitly omitted repository output and adjusted one repeated estimate from:

27.0% → approximately 23.9%

while the substantial reduction in model cycles and cached-input processing remained.

For that reason, I treat #35050 only as directional supporting evidence and do not rely on its original headline savings percentages for the main argument here.

The cleaner structural evidence for this issue is #41450.


7. Chronology rules out some tempting causal explanations

The chronology is important.

The release:

0.150.0-alpha.12.2

was tagged on Aug 27 and points to:

a9802304f60ab14c0b07e3ee0db9a9c105ab0cb3

The strongest #41541 comparison already observes elevated model-step density while using that version.

Two later changes therefore cannot explain that earlier observation.

Root service-tier inheritance

Commit:

dc2ccc6843abb09c9d297862dc10b6bd12a3935d

PR:

#41308 — Make subagents follow the root service tier

This landed on Aug 28.

Proactive multi-agent delegation guidance

Commit:

4ee04c0aa5833ac39b1763f6ea44c7bc777c83dd

PR:

#41380 — Clarify proactive multi-agent delegation guidance

This also landed on Aug 28.

The latter changes model-facing guidance to encourage delegation when parallelization could improve speed or quality, including from root agents and subagents.

These later changes may affect later execution behavior, but they cannot be used retroactively to explain the model-step-density observation already present with 0.150.0-alpha.12.2.

The safe chronology is:

Aug 27:
0.150.0-alpha.12.2 exists

        ↓

higher model-step density is already observable
in the #41541 comparison

        ↓

Aug 28:
service-tier inheritance and stronger
proactive delegation guidance land later

The later changes should therefore be considered separately rather than treated as the cause of the earlier telemetry.


8. A nearby accounting change should be distinguished from execution amplification

Commit:

4761851ff35c4ebdd35eb8801e1180a0a50fef60

PR:

#41183 — Account subagent token usage toward root goals

records descendant token usage toward the root goal's internal progress/budget.

This changes how already-occurring descendant usage is accounted for within the goal hierarchy.

By itself, it does not demonstrate creation of additional model requests.

It should therefore be distinguished from execution-graph changes.

Conceptually:

counting existing descendant work differently

is not the same mechanism as:

creating additional model cycles

What steps can reproduce the bug?

There are two separate evidence layers.

A. Current post-announcement/post-reset symptom

After the Aug 29 usage-efficiency announcement and paid-user usage reset, a fresh post-reset usage window still showed severe depletion:

more than 80 percentage points of weekly usage over roughly four hours of ordinary repository-development activity

This report intentionally does not publish private repository content, session identifiers, detailed activity history, or other project-specific information.

This observation establishes the current symptom after the Aug 29 announcement/reset boundary, but does not by itself identify its cause or establish the deployment state of every announced fix.

B. Controlled reproduction of a plausible amplification mechanism

The deterministic fixture reported in #41450 provides a controlled reproduction of the structural amplification mechanism.

A simplified reproduction is:

  1. Use the official codex-cli 0.150.0-alpha.12.2.
  2. Provide a large retained context.
  3. Define multiple already-known independent, read-only checks.
  4. Run one condition where the checks are serialized across separate outer model continuations.
  5. Run another condition where the same checks are grouped into bounded concurrent tool execution.
  6. Keep the required checks and nested tool workload unchanged.
  7. Compare:
    • outer model cycles;
    • cumulative logical input;
    • cached input;
    • final task coverage.

The reported primary result is:

Serial:
60 outer cycles
11,100,973 logical input
86 nested calls
50/50 checks

Batched:
15 outer cycles
2,041,915 logical input
86 nested calls
50/50 checks

This controlled reproduction does not prove that the current post-reset symptom is caused by this mechanism.

It demonstrates that the proposed amplification path is real and large enough to be technically relevant.


What is the expected behavior?

When multiple tool operations are already known to be independent, read-only, and safe to execute together, Codex should avoid unnecessary outer model re-entry between each operation.

Large retained contexts should not be repeatedly reprocessed solely because independent work is represented as:

model → tool → model → tool → model → tool

when the same bounded stage can safely execute with fewer outer model cycles.

Usage-efficiency improvements should reduce avoidable model work without leaving an execution pattern that can offset those savings through repeated large-context replay.


Additional information

What this evidence supports

The evidence presented here supports that:

  • a severe usage symptom was observable after the Aug 29 announcement/reset boundary;
  • recent measured 0.150.x sessions in #41541 show higher model-step density than the earlier comparison cohort;
  • additional outer model cycles over a large retained context can substantially amplify cumulative logical input;
  • that amplification can occur while preserving the same required tool workload and task coverage;
  • the recent publicly visible retained-image, memory-worker, and execution-failure fixes address technically different waste mechanisms.
What this evidence does not establish

The current evidence does not establish:

  • whether every announced Aug 29 fix had already been deployed to the runtime associated with the post-reset observation;
  • one specific commit responsible for the increased model-step density;
  • that the post-reset symptom is definitely caused by outer-cycle/context-replay amplification;
  • that the increased model steps in #41541 are specifically unnecessary serial tool re-entry;
  • that all Codex workloads exhibit the same amplification;
  • that the 5.4x controlled result generalizes beyond that fixture;
  • that Sol is intrinsically more susceptible than Terra or Luna;
  • that the later service-tier or delegation changes caused the earlier 0.150.0-alpha.12.2 observation.

The narrower concern is:

A severe usage symptom remained observable after the Aug 29 announcement/reset boundary. Separately, a reproducible outer-model-cycle/context-replay amplification mechanism exists, recent real-session telemetry shows increased model-step density, and the publicly visible recent fixes address different failure modes. This makes model-cycle/context-replay amplification a plausible contributor worth distinguishing from the previously announced waste mechanisms.

Related technical evidence
  • #41541 — recent-runtime model-step/token-throughput comparison
  • #41450 — controlled outer-cycle/context-replay amplification
  • #35050 — earlier batching experiments, with the later output-truncation caveat described above
  • #41369 — repeated large cached-context processing across sequential continuations
  • #35259 — model-mediated wait/status polling overhead
  • #34971 — repeated processing of large retained context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the controlled reproduction in #41450 using the official codex-cli 0.150.0-alpha.12.2 client, isolated CODEX_HOME, and deterministic fake Responses server. Compare its execution shape with the telemetry in #41541 and the public changes in #40994, #40587, and #41454. Done means determining whether unnecessary outer model re-entry over retained context is present and identifying a reproducible result or concrete code location.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.