anthropics / anthropics/claude-ai-mcp

[claude.ai web] MCP tool-result images are not in the model's context on the calling turn — visible one turn later

Abierto
#731 2 comentarios 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
Sin datos de lenguaje
Estrellas
471
Forks
76
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Companion report to #643, which covers Claude Desktop. **This is claude.ai web**, which #643 explicitly does not test. Reproduced with verifiable ground truth.

## Summary

An MCP tool returns a valid `image` content block. On the turn the tool returns, the model reports it as an empty placeholder. Asked again on the **next** turn, with no second tool call, the model sees the image and reads it correctly.

The tool call reports success, so the failure is silent.

## Environment

- claude.ai **web**, Opus 5, effort High
- Custom connector, remote Streamable HTTP, org-level
- Tool permission: Always allow
- Observed 2026-07-30

## Reproduction

The tool returns one `text` block and one `image` block:

```json
{"type": "image", "data": "", "mimeType": "image/png"}
```

Server-side the block was verified before sending: valid PNG magic bytes, 1100x850, base64 decodes byte-for-byte to the source, 108 KB — well inside the 5 MB and 20-image limits.

**Turn 1** — "call get_document …, then tell me the claim number, driver name, claimant name and total incurred":

> "no image content actually reached me. The image slot came through empty … So, plainly: I cannot see the page."

The `text` block from the *same* tool result arrived intact — it quoted the filename, page count and link back correctly.

**Turn 2** — "look again at the page image from that tool call, now, without calling the tool again":

> "Yes — I can see it now. That's a straight contradiction of what I told you last turn … the page image is present in my context in this turn, fully legible, and it wasn't reaching me when I answered before."

## Ground truth

The page is a one-page insurance loss run. Its contents were read directly from object storage before the test, so the turn-2 answer could be checked rather than trusted:

| Field | Ground truth | Turn 2 output |
|---|---|---|
| Claim # | `F3K8777` | `F3K8777` |
| Driver | `VICTOR M NIEVES, 49` | `Victor M Nieves, 49` |
| Second claimant | `SHELL GAS STATION` | `Shell Gas Station` |
| Loss reserve | `$1,000.00` | `$1,000.00` |
| Expense reserve | `$190.00` | `$190.00` |
| Total incurred | `$1,190.00` | `$1,190.00` |
| Examiner | `Amy Martin` | `Amy Martin` |

It additionally read a second claim on the page correctly. This is not confabulation — the filename contains only `WN405948` and `Northland`, and none of the values above are derivable from it. The model noted this itself on turn 1, warning that the filename "is a filename, not a reading of the document."

## Expected

The image should be in the model's context on the turn the tool result is delivered, as it is in Claude Code.

## Impact

Because the tool reports success and the accompanying text arrives, a model with no signal that anything is missing produces a *plausible* answer rather than a true one, or hedges in a way that misattributes the cause. In our case an agent asked to match insurance carriers fetched four loss runs and then described them as illegible — sending us to investigate render resolution for two days. The images were correct the whole time.

## Notes

- `text` blocks in the same tool result are unaffected.
- An embedded `resource` block with `mimeType: application/pdf` is separately rejected with "resources of type application/pdf are not currently supported" — consistent with the connector docs listing binary under `resources/read` rather than tool results. Mentioned only to rule it out as a workaround.
- Not tested on mobile.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.