openai / openai/codex

Repeated full-history requests can generate hundreds of GB of redundant client upload per month

Open
#45,971 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug connectivity context
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What happened

A long-running, image-heavy Codex session lost the incremental Responses WebSocket path and began
submitting its full active history over HTTP. Local transport instrumentation measured serialized
request bodies of approximately 22.4–22.6 MiB. Normal model/tool loops could submit that body
multiple times per minute.

Independent UniFi application traffic totals attributed the following traffic to ChatGPT, almost
entirely from the affected workstation:

Window Total traffic Upload Download
24 hours 69.0 GB 65.9 GB 3.12 GB
7 days 309 GB 292 GB 17.2 GB
Rolling 30 days 427 GB 386 GB 40.9 GB

The workstation accounted for 99.8% of the ChatGPT-classified traffic in the 30-day view. One
five-minute bucket contained approximately 878 MB of traffic.

During one clearly correlated 35-minute active interval, the router recorded 1.50 GiB total / 1.41
GiB upload
—roughly the volume of sixty full 22.6 MiB submissions. In the 45 minutes after the
mitigation below was applied, traffic fell to 27.1 MiB total / 14.0 MiB upload, while work in the
session continued.

The attached 24-hour router graph shows the pattern clearly: sustained work, an idle night, a short
return to work using the old request path, and the drop immediately after the mitigation was
applied.

Image

Why this may matter to OpenAI

The connection is unlimited, so this did not create a meaningful bandwidth cost for the user. It
also did not materially concern the ISP. The unusually large recipient-side cost is easy to miss:
OpenAI still receives and handles every repeated full-history body.

I cannot estimate OpenAI's internal bandwidth pricing, and raw cloud ingress may be free or
discounted. But TLS termination, routing, buffering, deserialization, request processing, and
operational capacity are not made free by the user's unlimited connection. If the observed 65.9
GB/day
rate were sustained, it would be approximately 2 TB/month from one workstation.

This is therefore not only a latency or metered-connection problem. It is silent provider-side
resource amplification at fleet scale.

Mitigation that worked

The successful mitigation was:

  1. Compact the active history with caller-authored guidance.
  2. Omit inline image bodies from the installed compacted working history, while leaving the
    source transcript unchanged and recoverable.
  3. Only after that successful size reduction, allow one explicit attempt to establish a fresh
    Responses WebSocket continuation.
  4. If that attempt fails, return to sticky HTTP fallback rather than automatically retrying the
    same oversized full body.

The first full request on the fresh WebSocket was approximately 275 KiB instead of 22.6 MiB—an
approximately 84x reduction. Following turns used incremental request items.

The important properties appear to be:

  • account for retained media in serialized request bytes, not only model tokens;
  • avoid retrying a deterministic oversized full request several times;
  • prevent HTTP fallback from turning every model/tool loop into another full-history upload;
  • allow a successfully reduced context to re-enter the incremental path without restarting the
    thread;
  • keep the original transcript available for inspection and restoration.

Reproduction shape

  1. Use a long-running Codex thread containing multiple screenshots or image-returning tool calls.
  2. Accumulate enough inline media in active or compacted history to make a full serialized request
    tens of megabytes even though token accounting remains below the compaction threshold.
  3. Lose incremental WebSocket reuse and enter HTTP fallback.
  4. Continue through several model/tool loops.
  5. Observe repeated multi-megabyte full-history submissions and rapidly growing client upload.

The exact trigger for losing WebSocket reuse may vary. The expensive behavior is what happens after
the client no longer has a valid incremental continuation.

Expected behavior

  • Normal model/tool continuations should not repeatedly upload the complete accumulated history.
  • Inline media bytes should participate in compaction and request-size thresholds.
  • The client should avoid multiple retries of the same deterministic oversized request.
  • Fallback should remain stable until the context has materially changed or been reduced.
  • Codex should expose serialized request size, inline-media bytes, full-versus-incremental mode,
    transport fallback, and cumulative bytes resent.
  • A configurable request/rolling-session byte guardrail would keep one thread from silently
    generating hundreds of gigabytes of upload.

Environment

  • Linux x86_64.
  • ChatGPT Pro subscription.
  • gpt-5.6-sol.
  • Custom Codex fork based on rust-v0.154.0-alpha.4.
  • The fork added request-size/transport-state observability and the mitigation described above.

The upstream mechanism has independent reports in #45109, #24550, #43015, and #38014. This datapoint
adds the router-scale aggregate, a continued-work before/after comparison, and a mitigation that
restored small incremental requests.

The raw rollout is not attached because it contains prompts, tool output, local paths, and image
payloads. No session IDs, authentication data, or private content are included here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by tracing the compaction, serialized request-size accounting, Responses WebSocket continuation, and HTTP fallback paths described in the issue, then reproduce the long-running image-heavy history shape. Done should prevent repeated oversized full-history uploads, preserve recoverable transcript data, restore incremental continuation after reduction, and expose the requested transport and byte metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, networking, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.