code-yeongyu / code-yeongyu/senpi
Tool-result image data URLs are double-prefixed for Codex Responses, producing invalid image_url
- Dominant language
- TypeScript
- Stars
- 429
- Forks
- 98
- Avg merge
- 5h 3m
- Merged PRs (30d)
- 526
Description
## Summary
Image-producing tools can return image content whose `data` field is already a `data:image/...;base64,...` URL. The OpenAI/Codex Responses converter unconditionally adds another data URL prefix, producing an invalid payload:
```text
data:image/png;base64,data:image/png;base64,...
```
Codex then rejects the request with:
```text
Invalid `input[214].output[1].image_url`. Expected a base64-encoded data URL with an image MIME type, but got an invalid base64-encoded value.
```
## Environment
- `omo-ai`: `5.0.0-0.beta.31`
- `@code-yeongyu/senpi`: `2026.8.31`
- `@code-yeongyu/senpi-ai`: `2026.8.31`
- Provider API: `openai-codex-responses`
- OS: Linux x64
- Node.js: 24.x
## Root-cause evidence
`convertToolResultOutput()` currently constructs every tool-result image as:
```ts
image_url: `data:${image.mimeType};base64,${image.data}`
```
This works when `image.data` is raw base64, but double-prefixes data URLs.
I reproduced this locally against the installed converter using the same valid 1x1 PNG payload in both representations:
- raw base64 input -> valid data URL
- `data:image/png;base64,` input -> `data:image/png;base64,data:image/png;base64,` -> invalid
The `input[*].output[*].image_url` parameter also identifies this as a tool-call output image rather than a normal user attachment.
## Expected behavior
The tool-result boundary should establish one canonical representation:
- accept raw base64 and add exactly one prefix; or
- detect a valid image data URL and preserve/normalize it; or
- reject non-canonical image content before it is persisted and replayed.
Malformed image data should not permanently poison subsequent turns.
## Suggested regression tests
1. Raw base64 tool-result images serialize to exactly one data URL prefix.
2. Already-prefixed data URLs do not become double-prefixed.
3. Malformed or empty image payloads are rejected or converted to a text placeholder before provider submission.
4. A failed image tool result does not wedge later plain-text turns.
## Related but not duplicate
- #302 concerns cumulative image bytes causing HTTP 413.
- openai/codex#16605 concerns empty clipboard image payloads.
- openai/codex#32287 concerns cumulative retained-image limits after compaction.
Contributor guide
Research direction
Find the TypeScript implementation of convertToolResultOutput() in the OpenAI/Codex Responses converter and inspect how tool-result images are serialized. Add regression coverage for raw base64 and already-prefixed data URLs, then verify malformed or empty payload behavior and that later plain-text turns still submit successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100