perf: original-detail image estimator repeatedly decodes every image above its 32-entry cache capacity

Summary

The original-detail image token estimator uses a process-global 32-entry LRU cache. Repeated sequential scans of 33 distinct images miss on every lookup, causing all 33 images to be fully decoded again just to recover their dimensions.

I reproduced this using the upstream estimator/cache function bodies and the actual upstream image-preparation utility. This is a component-level reproduction, not an end-to-end Codex latency measurement.

Version and environment

• Source: a5c15ab5c0ae04bef3464d5bbaa7c1c70cc8b253
• Relevant estimator/cache files remained unchanged on aaa2cabfbcb8d9997ce67e166f796f46d5b72342 on September 15.
• Apple M4, 16 GiB RAM, macOS 15.6.1, arm64
• Rust 1.94.1, release build; dependency versions aligned with Codex's lockfile

Reproduction

Extract the attached ZIP, enter its directory, then run:

cargo run --release --locked -- --preparation > baseline.csv
PROBE_CACHE_CAPACITY=64 target/release/codex-image-estimator-probe --preparation > control.csv

The executable generates synthetic PNG/JPEG/WebP images, prepares them through PromptImageMode::ORIGINAL_DETAIL, then repeatedly estimates their token cost. No credentials, model calls, GPU or private histories are required.

All prepared image URLs remain byte-identical. Both configurations return identical estimates, matching calculations from the dimensions already returned by image preparation.

For 33 synthetic 2048×2048 PNGs, the original isolated benchmark measured median repeat-scan times of 558–621 ms across two experiments, versus approximately 2.4 ms with capacity 64. A follow-up through actual upstream preparation reproduced the behavior.

Timings exclude fixture generation/preparation and are specific to these synthetic inputs and this machine. Increasing capacity only moves the threshold; it is not a complete fix.

Root cause and possible direction

The miss path fully decodes image data:
https://github.com/openai/codex/blob/a5c15ab5c0ae04bef3464d5bbaa7c1c70cc8b253/codex-rs/core/src/context_manager/history.rs#L1000

The cache also holds its shared mutex while executing the miss factory:
https://github.com/openai/codex/blob/a5c15ab5c0ae04bef3464d5bbaa7c1c70cc8b253/codex-rs/utils/cache/src/lib.rs#L30

Image preparation already computes the dimensions:
https://github.com/openai/codex/blob/a5c15ab5c0ae04bef3464d5bbaa7c1c70cc8b253/codex-rs/core/src/image_preparation.rs#L306

Retaining validated dimensions or estimates with the image/history lifecycle could avoid this repeated work. That needs correct handling of resizing, detail, resume and memory limits.

Production frequency, concurrent-session contention and aggregate task-completion impact remain unmeasured. This report does not attribute existing multi-minute stalls to this mechanism or claim inference-token savings.

Codex-image-cache-repro.zip

Run the supplied reproduction with the 32-entry and 64-entry controls first. Read codex-rs/core/src/context_manager/history.rs, codex-rs/utils/cache/src/lib.rs, and codex-rs/core/src/image_preparation.rs to trace the miss path, locking, and existing dimensions. Done means repeated scans avoid unnecessary full decodes while preserving resizing, detail, resume, and memory-limit behavior.