aaif-goose / aaif-goose/goose

Many-image dimension validation errors permanently strand sessions instead of triggering compaction

Open
#11,871 2 comments 0 reactions 1 assignee Claimed by @DOsinga View on GitHub
Dominant language
Rust
Stars
54.2k
Forks
6.2k
Avg merge
3d 4h
Merged PRs (30d)
240

Description

🤖 Filed by Morgan's AI agent from a reproduced user report.

## Summary

A long-running, image-heavy Goose session becomes permanently unusable when Anthropic's conditional limit for many-image requests is triggered.

When a request contains more than 20 image blocks, Anthropic requires every image in the replayed request to have neither dimension exceed 2000 px. A historical image that was valid when originally sent can therefore become invalid later, once enough additional images accumulate.

Goose classifies the resulting HTTP 400 as a generic `RequestFailed`, retries the unchanged invalid request three times, and does not invoke its existing compaction recovery. Because every subsequent turn replays the same model-visible image history, even text-only follow-up messages fail permanently.

## Observed error

```text
Bad request (400):
messages.38.content.1.image.source.base64.data:
At least one of the image dimensions exceed max allowed size for many-image requests: 2000 pixels
```

The unchanged request was retried three times. A later retry identified another historical image at `messages.74.content.1` and failed with the same validation error.

## Reproduction evidence

The affected persisted session contained:

- 41 image blocks across user attachments and tool responses
- two historical images above the conditional limit: 1731×2048 and 2048×1338
- approximately 398k current input tokens against a configured 1.05M-token context limit

The session therefore had not reached its token-based auto-compaction threshold. The image-count threshold activated first.

The images were accepted on earlier turns. The failure appeared only after the request accumulated more than 20 images, at which point the stricter 2000 px rule applied to every historical image in the replay.

## Steps to reproduce

1. Start a Goose session using an Anthropic vision-capable model.
2. Add an image with one dimension between 2001 and 8000 px; it is valid while the request contains 20 or fewer images.
3. Continue the session until its model-visible history contains more than 20 image blocks. Images nested in tool results count as well.
4. Send another message, including a text-only message.
5. Observe the `many-image requests: 2000 pixels` HTTP 400.
6. Send another text-only message and observe the same failure because Goose replays the invalid historical image.

Anthropic documents this conditional limit here:
https://docs.anthropic.com/en/docs/vision#request-limits

## Expected behavior

A provider validation error caused by accumulated model-visible history must not permanently disable the session.

For this deterministic error, Goose should use the same recovery principle as byte-size request overflow:

- recognize the `many-image requests` dimension error as recoverable through context compaction;
- avoid retrying an identical invalid payload unchanged;
- compact the conversation so historical image bytes become text placeholders, then retry once;
- preserve the original images in user-visible session history while removing them from model-visible replay.

A longer-term preflight based on model-visible image count and dimensions could avoid the rejected request, but classifying this provider response for the existing compaction path would close the immediate unrecoverable-session failure.

## Related issues

- #11171 — byte-size request-limit errors were similarly leaving image-heavy sessions permanently stuck; fixed by #11173 through error classification and compaction.
- #6714 — accumulated image payload caused later requests to exceed a request-size limit.
- #11260 — persisted invalid history caused every subsequent replay to fail deterministically.

Contributor guide

Open the contributing guide

Research direction

Look at the existing compaction recovery logic from PR #11173, likely in the request handling or error classification module. Identify where HTTP 400 errors are processed and how the 'many-image requests' dimension error can be classified as recoverable. The fix involves modifying the error handler to trigger compaction for this specific error, similar to byte-size overflow handling. Test by reproducing the session failure with many images.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.