History Notes forwards `detail: original` when the selected model does not support it
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Background
History Notes can return image attachments from the Codex history backend. These attachments are converted into input_image function-call output items and included in the following Responses request.
The output path preserves detail: "original" even when the selected model has supports_image_detail_original = false.
Affected version
- Latest
mainrevision tested:1c1e17782aeb51a5a253997067fa887a9d593cc9 - Codex CLI:
0.151.0-alpha.7.2 - OS: Windows 11 Pro for Workstations
10.0.22631, x64
Minimal reproduction
Adapt the existing integration test in codex-rs/ext/history-notes/tests/image_model_input.rs:
-
Configure the selected model as not supporting original-detail images:
.with_model_info_override("gpt-5.4", |model_info| { model_info.supports_image_detail_original = false; }) -
Keep the mocked History Notes backend response containing:
{ "data": "<valid 1x1 PNG base64>", "mime_type": "image/png", "detail": "original" } -
Expect the next Responses request to contain
detail: "high", matching the existingsanitize_original_image_detailbehavior for models without original-detail support. -
Run:
just test -p codex-history-notes-extension history_images_reach_the_next_model_request
The assertion fails consistently because the actual outbound item retains detail: "original":
- "detail": "original"
+ "detail": "high"
The regression assertion failed on both nextest attempts. After restoring the test, the existing crate suite passed: 9 tests run, 9 passed.
Expected behavior
History Notes tool images should follow the selected model's image-detail capability. When supports_image_detail_original is false, detail: "original" should be normalized to the default supported detail (high) before the next Responses request is serialized.
Actual behavior
The next Responses request contains the History Notes image with detail: "original" despite the selected model declaring that original detail is unsupported.
This produces an outbound request that violates the selected model's advertised capability and can make History Notes turns fail on models or providers that reject original-detail image input.
Root cause
HistoryNotesToolOutput::new in codex-rs/ext/history-notes/src/tools.rs copies the backend's image detail directly into FunctionCallOutputContentItem::InputImage:
content.push(FunctionCallOutputContentItem::InputImage {
image_url: format!("data:{mime_type};base64,{data}"),
detail,
});
HistoryNotesToolOutput has no model capability context, and its to_response_item implementation returns the payload unchanged.
The general history normalization path strips images for models without image input, but it does not normalize unsupported original detail for image-capable models.
Equivalent model-aware normalization already exists:
codex_tools::sanitize_original_image_detail- MCP output handling passes
original_image_detail_supportedand sanitizes before producing the response payload. - Code Mode sanitizes runtime image detail using
can_request_original_image_detail(turn.model_info()).
The History Notes path bypasses these existing safeguards.
Impact
The issue affects token-budget History Notes sessions when all of the following are true:
- The history backend returns an image with
detail: "original". - The selected model supports image input but does not support original-detail images.
- The model invokes a History Notes tool that returns that attachment.
The affected request is generated deterministically. The scope is limited to History Notes and models without original-detail support.
Potential fix direction
Apply sanitize_original_image_detail to extension tool outputs before they enter history, using the selected turn's ModelInfo. Centralizing this at the extension-tool dispatch boundary would prevent similar capability mismatches in other extension outputs.
Regression coverage should verify:
originalis downgraded for an image-capable model withsupports_image_detail_original = false.originalis preserved for a model that supports it.- Existing text-only image stripping remains unchanged.
Duplicate review
I searched open and closed Issues, open/draft/closed/merged PRs, recent commits, Discussions, and release-note text using:
HistoryNotesToolOutputhistory notes image detailhistory backend imagesupports_image_detail_original historysanitize_original_image_detail historyunsupported original image detail
Related but different records:
- #40595 concerns clipboard images being discarded for text-only models. History Notes images are already stripped by the general history normalizer for text-only models.
- #24797 concerns remote compaction rejecting the
autovariant in persisted history. - #19806 concerns token/byte estimation for original-detail images.
- #41292 introduced History Notes image forwarding but does not test a model with original-detail support disabled.
No existing Issue, PR, commit, or Discussion was found for this History Notes capability mismatch.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with codex-rs/ext/history-notes/src/tools.rs and the existing integration test in codex-rs/ext/history-notes/tests/image_model_input.rs. Read codex_tools::sanitize_original_image_detail and the extension-tool dispatch path to understand where model capability information is available. Run just test -p codex-history-notes-extension history_images_reach_the_next_model_request; done means unsupported original detail is downgraded, supported detail is preserved, and text-only image stripping still passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100