code-yeongyu / code-yeongyu/senpi

Tool-result image data URLs are double-prefixed for Codex Responses, producing invalid image_url

Open
#1,260 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
429
Forks
98
Avg merge
5h 3m
Merged PRs (30d)
526

Description

## Summary

Image-producing tools can return image content whose `data` field is already a `data:image/...;base64,...` URL. The OpenAI/Codex Responses converter unconditionally adds another data URL prefix, producing an invalid payload:

```text
data:image/png;base64,data:image/png;base64,...
```

Codex then rejects the request with:

```text
Invalid `input[214].output[1].image_url`. Expected a base64-encoded data URL with an image MIME type, but got an invalid base64-encoded value.
```

## Environment

- `omo-ai`: `5.0.0-0.beta.31`
- `@code-yeongyu/senpi`: `2026.8.31`
- `@code-yeongyu/senpi-ai`: `2026.8.31`
- Provider API: `openai-codex-responses`
- OS: Linux x64
- Node.js: 24.x

## Root-cause evidence

`convertToolResultOutput()` currently constructs every tool-result image as:

```ts
image_url: `data:${image.mimeType};base64,${image.data}`
```

This works when `image.data` is raw base64, but double-prefixes data URLs.

I reproduced this locally against the installed converter using the same valid 1x1 PNG payload in both representations:

- raw base64 input -> valid data URL
- `data:image/png;base64,` input -> `data:image/png;base64,data:image/png;base64,` -> invalid

The `input[*].output[*].image_url` parameter also identifies this as a tool-call output image rather than a normal user attachment.

## Expected behavior

The tool-result boundary should establish one canonical representation:

- accept raw base64 and add exactly one prefix; or
- detect a valid image data URL and preserve/normalize it; or
- reject non-canonical image content before it is persisted and replayed.

Malformed image data should not permanently poison subsequent turns.

## Suggested regression tests

1. Raw base64 tool-result images serialize to exactly one data URL prefix.
2. Already-prefixed data URLs do not become double-prefixed.
3. Malformed or empty image payloads are rejected or converted to a text placeholder before provider submission.
4. A failed image tool result does not wedge later plain-text turns.

## Related but not duplicate

- #302 concerns cumulative image bytes causing HTTP 413.
- openai/codex#16605 concerns empty clipboard image payloads.
- openai/codex#32287 concerns cumulative retained-image limits after compaction.

Contributor guide

Open the contributing guide

Research direction

Find the TypeScript implementation of convertToolResultOutput() in the OpenAI/Codex Responses converter and inspect how tool-result images are serialized. Add regression coverage for raw base64 and already-prefixed data URLs, then verify malformed or empty payload behavior and that later plain-text turns still submit successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.