openai / openai/codex

Repeated image-context uploads cause network-wide latency spikes; throttling slows Codex steps

Open
#45,109 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app bug connectivity context performance windows-os
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Repeated image-context uploads cause connection-wide latency spikes; throttling trades gaming latency for slower Codex steps

Summary

On Windows, an image-assisted Codex task repeatedly sent 11–16.5 MB compressed
requests. Process-attributed Windows network tracing and simultaneous pings
show that these uploads coincided with internet RTT rising from about 9 ms to
105–137 ms, plus occasional 900-ms timeouts. The local Ethernet gateway stayed
below 1 ms. Valorant was affected while Codex worked in the background.

A temporary 10-Mbit/s upload limit substantially improved RTT during a real
15-MB Codex upload, but increased the time spent transferring each request.
The user subsequently reported very slow steps in two tasks, removed the
limit, and reported that the network spikes returned. Only one of those two
tasks was identified and inspected; the other was not independently diagnosed.

Please investigate repeated transmission of accumulated tool images and
provide a supported way to bound serialized image-context size while retaining
useful task state. Bandwidth throttling alone is an unsatisfactory tradeoff.

Environment

  • Windows, wired gigabit Ethernet; actual WAN capacity was not independently measured.
  • Official Codex desktop package 26.908.4834.0.
  • Bundled CLI embedded version marker: 0.154.0-alpha.6.2; active executables
    matched the official package mirror. No custom binary was active.
  • Affected task: gpt-6-astra, reasoning effort ultra, ChatGPT authentication.
  • Actual measured requests used responses_http.
  • Request compression and compaction_image_budget were enabled.
  • Non-default setting: model_auto_compact_token_limit_scope = "body_after_prefix".
  • Tests performed September 12, 2026. Times below are UTC; MB means decimal MB.

Measured uploads and image-context growth

Application logs recorded:

Request time JSON bytes before zstd Bytes after zstd
19:14:43 15,127,472 11,141,163
19:15:02 16,795,345 12,391,935
19:15:29 20,210,291 14,956,043

The three compressed bodies total 38,489,141 bytes. Independently, ETW
attributed about 38.62 MB sent during the one-minute window to codex.exe,
96.90% of captured TCP/UDP send bytes; Valorant accounted for 1.203 MB.

Structural inspection of the affected history through 19:15:35 found 13 inline
image occurrences totaling 19,567,858 data-URL characters after the last
replacement history at 18:58:26. All came from new custom_tool_call_output
items after that compaction. This closely explains the roughly 20.2-MB JSON
request, but is not a captured/decrypted HTTP body or exact image deduplication
analysis. The evidence does not establish that all those images were obsolete.

Runtime logs showed about 99,854–106,271 tokens relevant to auto-compaction,
against a 244,800 threshold. Substantial image-byte growth therefore occurred
without reaching the configured token threshold. zstd was already working;
enabling compression again would not fix the issue.

Network mitigation experiment

We compared two approximately 60-second windows, using parallel pings to the
gateway, 1.1.1.1 and 8.8.8.8, adapter byte counters, and the
Microsoft-Windows-Kernel-Network ETW provider. Attribution used the event
payload PID for TCP/UDP send events 10/26/42/58, not generic process I/O counters
or the kernel execution PID. No speed-test traffic was generated.

The intervention was a temporary Windows QoS rule matching codex.exe, TCP
destination port 443, at 10,000,000 bit/s.

Observation Before With limit
Ping samples 82 84
Successful Cloudflare RTTs 9–137 ms 9–14 ms
Mean successful Cloudflare RTT while adapter upload exceeded 0.5 MB/s 86.80 ms, 10 samples 10.41 ms, 17 samples
Samples where both public targets timed out at 900 ms 3 1
Peak total adapter upload 40.28 Mbit/s 10.57 Mbit/s

The after-window included a newly logged 15,018,283-byte compressed request and
15,063,151 Codex-attributed send bytes. It was not an idle-network comparison.
Gateway replies remained below 1 ms in both windows. Failed pings were excluded
from RTT statistics, rather than interpreted as their recorded zero-ms value.

This strongly supports upload-induced WAN queueing, but router queues were not
directly instrumented. The windows contain different request counts and only
one minute each; they do not establish a reliable packet-loss rate or complete
elimination of disruptions.

Subsequent slowdown and rollback

A later image-assisted task still sent about 16–16.5 MB per request. Several
logged request-start-to-HTTP-200 intervals were about 15 seconds. That interval
includes upload and server time; it is not a pure upload measurement or a
controlled estimate of the limit's slowdown. At 10 Mbit/s, transferring 16.5 MB
alone takes approximately 13.2 seconds in the ideal single-flow case.

That task also logged incremental request failed, websocket reuse properties didn't match, followed later by falling back to HTTP. Successful HTTP
responses and subsequent sampling continued, so this was not evidence of a
permanent hang. The cause of the WebSocket failure was not established.

The user confirmed running the rollback and reported renewed ping spikes.
The rollback/recurrent-spike report was not followed by a fresh ETW comparison.
Manual /compact was suggested but could not be tried at that time; no
post-compaction improvement is claimed. No task history was edited.

Requested improvements

  1. Bound aggregate image-context bytes independently of token-based compaction,
    with safe summarization/eviction of old tool images and preservation of
    references needed for the task.
  2. Investigate repeated full image-history uploads on the HTTP fallback path;
    expose when incremental reuse is unavailable and why.
  3. Show request size and distinguish uploading, awaiting server response, and
    tool execution so long steps are understandable.
  4. Offer upload pacing or bandwidth controls as an optional mitigation, without
    treating them as a substitute for reducing redundant context transfer.

Related: https://github.com/openai/codex/issues/43015 describes large
image-history requests despite zstd. This report adds independent
process-attributed network measurements and a measured throttling tradeoff.
It does not assume the two cases have an identical internal root cause.

This report contains aggregate measurements and selected non-secret log fields.
Private prompts, images, task IDs, local paths, authentication data, raw traces,
and the full conversation are deliberately not attached.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the responses_http request path, auto-compaction settings, and the HTTP fallback behavior described in this report; compare how accumulated tool images are selected and transmitted. Use issue 43015 as related context, then verify that image-context size can be bounded, redundant uploads are reduced, and request upload, server wait, and tool execution are distinguishable.

Written by the indexing model from the issue text.

Assessment

Domain
cli, networking, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.