aaif-goose / aaif-goose/goose

Many-image dimension validation errors permanently strand sessions instead of triggering compaction

Open
#11,871 2 comments 0 reactions 1 assignee Claimed by @DOsinga View on GitHub
Dominant language
Rust
Stars
54.2k
Forks
6.2k
Avg merge
3d 2h
Merged PRs (30d)
262

Description

🤖 Filed by Morgan's AI agent from a reproduced user report.

## Summary

A long-running, image-heavy Goose session becomes permanently unusable when Anthropic's conditional limit for many-image requests is triggered.

When a request contains more than 20 image blocks, Anthropic requires every image in the replayed request to have neither dimension exceed 2000 px. A historical image that was valid when originally sent can therefore become invalid later, once enough additional images accumulate.

Goose classifies the resulting HTTP 400 as a generic `RequestFailed`, retries the unchanged invalid request three times, and does not invoke its existing compaction recovery. Because every subsequent turn replays the same model-visible image history, even text-only follow-up messages fail permanently.

## Observed error

```text
Bad request (400):
messages.38.content.1.image.source.base64.data:
At least one of the image dimensions exceed max allowed size for many-image requests: 2000 pixels
```

The unchanged request was retried three times. A later retry identified another historical image at `messages.74.content.1` and failed with the same validation error.

## Reproduction evidence

The affected persisted session contained:

- 41 image blocks across user attachments and tool responses
- two historical images above the conditional limit: 1731×2048 and 2048×1338
- approximately 398k current input tokens against a configured 1.05M-token context limit

The session therefore had not reached its token-based auto-compaction threshold. The image-count threshold activated first.

The images were accepted on earlier turns. The failure appeared only after the request accumulated more than 20 images, at which point the stricter 2000 px rule applied to every historical image in the replay.

## Steps to reproduce

1. Start a Goose session using an Anthropic vision-capable model.
2. Add an image with one dimension between 2001 and 8000 px; it is valid while the request contains 20 or fewer images.
3. Continue the session until its model-visible history contains more than 20 image blocks. Images nested in tool results count as well.
4. Send another message, including a text-only message.
5. Observe the `many-image requests: 2000 pixels` HTTP 400.
6. Send another text-only message and observe the same failure because Goose replays the invalid historical image.

Anthropic documents this conditional limit here:
https://docs.anthropic.com/en/docs/vision#request-limits

## Expected behavior

A provider validation error caused by accumulated model-visible history must not permanently disable the session.

For this deterministic error, Goose should use the same recovery principle as byte-size request overflow:

- recognize the `many-image requests` dimension error as recoverable through context compaction;
- avoid retrying an identical invalid payload unchanged;
- compact the conversation so historical image bytes become text placeholders, then retry once;
- preserve the original images in user-visible session history while removing them from model-visible replay.

A longer-term preflight based on model-visible image count and dimensions could avoid the rejected request, but classifying this provider response for the existing compaction path would close the immediate unrecoverable-session failure.

## Related issues

- #11171 — byte-size request-limit errors were similarly leaving image-heavy sessions permanently stuck; fixed by #11173 through error classification and compaction.
- #6714 — accumulated image payload caused later requests to exceed a request-size limit.
- #11260 — persisted invalid history caused every subsequent replay to fail deterministically.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.