Repeated image-context uploads cause network-wide latency spikes; throttling slows Codex steps
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Repeated image-context uploads cause connection-wide latency spikes; throttling trades gaming latency for slower Codex steps
Summary
On Windows, an image-assisted Codex task repeatedly sent 11–16.5 MB compressed
requests. Process-attributed Windows network tracing and simultaneous pings
show that these uploads coincided with internet RTT rising from about 9 ms to
105–137 ms, plus occasional 900-ms timeouts. The local Ethernet gateway stayed
below 1 ms. Valorant was affected while Codex worked in the background.
A temporary 10-Mbit/s upload limit substantially improved RTT during a real
15-MB Codex upload, but increased the time spent transferring each request.
The user subsequently reported very slow steps in two tasks, removed the
limit, and reported that the network spikes returned. Only one of those two
tasks was identified and inspected; the other was not independently diagnosed.
Please investigate repeated transmission of accumulated tool images and
provide a supported way to bound serialized image-context size while retaining
useful task state. Bandwidth throttling alone is an unsatisfactory tradeoff.
Environment
- Windows, wired gigabit Ethernet; actual WAN capacity was not independently measured.
- Official Codex desktop package
26.908.4834.0. - Bundled CLI embedded version marker:
0.154.0-alpha.6.2; active executables
matched the official package mirror. No custom binary was active. - Affected task:
gpt-6-astra, reasoning effortultra, ChatGPT authentication. - Actual measured requests used
responses_http. - Request compression and
compaction_image_budgetwere enabled. - Non-default setting:
model_auto_compact_token_limit_scope = "body_after_prefix". - Tests performed September 12, 2026. Times below are UTC; MB means decimal MB.
Measured uploads and image-context growth
Application logs recorded:
| Request time | JSON bytes before zstd | Bytes after zstd |
|---|---|---|
| 19:14:43 | 15,127,472 | 11,141,163 |
| 19:15:02 | 16,795,345 | 12,391,935 |
| 19:15:29 | 20,210,291 | 14,956,043 |
The three compressed bodies total 38,489,141 bytes. Independently, ETW
attributed about 38.62 MB sent during the one-minute window to codex.exe,
96.90% of captured TCP/UDP send bytes; Valorant accounted for 1.203 MB.
Structural inspection of the affected history through 19:15:35 found 13 inline
image occurrences totaling 19,567,858 data-URL characters after the last
replacement history at 18:58:26. All came from new custom_tool_call_output
items after that compaction. This closely explains the roughly 20.2-MB JSON
request, but is not a captured/decrypted HTTP body or exact image deduplication
analysis. The evidence does not establish that all those images were obsolete.
Runtime logs showed about 99,854–106,271 tokens relevant to auto-compaction,
against a 244,800 threshold. Substantial image-byte growth therefore occurred
without reaching the configured token threshold. zstd was already working;
enabling compression again would not fix the issue.
Network mitigation experiment
We compared two approximately 60-second windows, using parallel pings to the
gateway, 1.1.1.1 and 8.8.8.8, adapter byte counters, and the
Microsoft-Windows-Kernel-Network ETW provider. Attribution used the event
payload PID for TCP/UDP send events 10/26/42/58, not generic process I/O counters
or the kernel execution PID. No speed-test traffic was generated.
The intervention was a temporary Windows QoS rule matching codex.exe, TCP
destination port 443, at 10,000,000 bit/s.
| Observation | Before | With limit |
|---|---|---|
| Ping samples | 82 | 84 |
| Successful Cloudflare RTTs | 9–137 ms | 9–14 ms |
| Mean successful Cloudflare RTT while adapter upload exceeded 0.5 MB/s | 86.80 ms, 10 samples | 10.41 ms, 17 samples |
| Samples where both public targets timed out at 900 ms | 3 | 1 |
| Peak total adapter upload | 40.28 Mbit/s | 10.57 Mbit/s |
The after-window included a newly logged 15,018,283-byte compressed request and
15,063,151 Codex-attributed send bytes. It was not an idle-network comparison.
Gateway replies remained below 1 ms in both windows. Failed pings were excluded
from RTT statistics, rather than interpreted as their recorded zero-ms value.
This strongly supports upload-induced WAN queueing, but router queues were not
directly instrumented. The windows contain different request counts and only
one minute each; they do not establish a reliable packet-loss rate or complete
elimination of disruptions.
Subsequent slowdown and rollback
A later image-assisted task still sent about 16–16.5 MB per request. Several
logged request-start-to-HTTP-200 intervals were about 15 seconds. That interval
includes upload and server time; it is not a pure upload measurement or a
controlled estimate of the limit's slowdown. At 10 Mbit/s, transferring 16.5 MB
alone takes approximately 13.2 seconds in the ideal single-flow case.
That task also logged incremental request failed, websocket reuse properties didn't match, followed later by falling back to HTTP. Successful HTTP
responses and subsequent sampling continued, so this was not evidence of a
permanent hang. The cause of the WebSocket failure was not established.
The user confirmed running the rollback and reported renewed ping spikes.
The rollback/recurrent-spike report was not followed by a fresh ETW comparison.
Manual /compact was suggested but could not be tried at that time; no
post-compaction improvement is claimed. No task history was edited.
Requested improvements
- Bound aggregate image-context bytes independently of token-based compaction,
with safe summarization/eviction of old tool images and preservation of
references needed for the task. - Investigate repeated full image-history uploads on the HTTP fallback path;
expose when incremental reuse is unavailable and why. - Show request size and distinguish uploading, awaiting server response, and
tool execution so long steps are understandable. - Offer upload pacing or bandwidth controls as an optional mitigation, without
treating them as a substitute for reducing redundant context transfer.
Related: https://github.com/openai/codex/issues/43015 describes large
image-history requests despite zstd. This report adds independent
process-attributed network measurements and a measured throttling tradeoff.
It does not assume the two cases have an identical internal root cause.
This report contains aggregate measurements and selected non-secret log fields.
Private prompts, images, task IDs, local paths, authentication data, raw traces,
and the full conversation are deliberately not attached.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the responses_http request path, auto-compaction settings, and the HTTP fallback behavior described in this report; compare how accumulated tool images are selected and transmitted. Use issue 43015 as related context, then verify that image-context size can be bounded, redundant uploads are reduced, and request upload, server wait, and tool execution are distinguishable.
Written by the indexing model from the issue text.
Assessment
- Domain
- cli, networking, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100