anthropics / anthropics/claude-ai-mcp

[Messages API / mcp_servers] MCP tool-result images are replaced with a text placeholder before the model — never delivered, and not documented

Offen
#969 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Keine Sprachdaten
Sterne
471
Forks
76
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

## What happened?

Using the **Messages API MCP connector** (`mcp_servers` + `anthropic-beta: mcp-client-2025-11-20`) — no
Claude Code, no Claude Desktop, no claude.ai — an MCP tool returns a spec-compliant `image` content block.
The image never reaches the model. The connector replaces the block with a hard-coded text string:

```
[MCP tool returned an image (format: image/png) but the Anthropic API doesn't currently support image responses]
```

The tool call reports `is_error: false`, so the failure is silent to the caller.

This is a companion report to #643 (Claude Desktop) and #731 (claude.ai web). Both of those describe an
image that *is* transported but attaches one turn late. **This is a different surface and a different
mechanism:** on the raw API the image is not late — it is removed and substituted, and never arrives on
any subsequent turn.

## What did you expect to happen?

Either the image block is forwarded to the model, or — if that is genuinely unsupported — the limitation
is stated in the MCP connector documentation.

Neither is true today. The [MCP connector docs](https://platform.claude.com/docs/en/agents-and-tools/mcp-connector)
list exactly two limitations:

> * Of the feature set of the MCP specification, only tool calls are currently supported.
> * The server must be publicly exposed through HTTP (supports both Streamable HTTP and SSE transports).
> Local STDIO servers cannot be connected directly.

Images are not mentioned there or anywhere else on the page. The "MCP tool result block" example shows
only a `text` block, with no statement that `text` is the *only* type that survives. A developer has no
way to learn this before shipping.

It also reads as inconsistent with the same page's client-side helpers section, which documents
`mcpResourceToContent` mapping MCP resources into Claude API content blocks including image and document
blocks. So image content blocks are a supported Claude API type; it is specifically the hosted connector
path that discards them.

## Steps to reproduce

**1.** Run a minimal MCP server whose tool returns one `text` block and one `image` block. Dependency-free
Python — the PNG encoder is hand-written, so no SDK or image library is in the path:

```python
# tool result returned by tools/call
{"content": [
{"type": "text", "text": "text-token=TEXTPATH-CEDAR-08. The image block that follows "
"carries a different image-token; read it from the pixels."},
{"type": "image", "mimeType": "image/png", "data": ""}
]}
```

The PNG renders the word `PURPLE-RHINO-42` as pixels. It appears **nowhere** in any text the model
receives, so the word cannot be produced from priors — it can only be read by seeing the image.

**2.** Call the Messages API with the server declared:

```bash
curl https://api.anthropic.com/v1/messages \
-H "content-type: application/json" \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "anthropic-beta: mcp-client-2025-11-20" \
-d '{
"model": "claude-opus-5",
"max_tokens": 2000,
"messages": [{"role": "user", "content":
"Call `text_and_image` exactly once. Then report (1) the text-token, (2) the image-token read from the pixels, and (3) describe verbatim every content block the tool returned. If a token is unavailable say NOT VISIBLE. Do not guess."}],
"mcp_servers": [{"type": "url", "url": "https:///mcp", "name": "image-probe"}],
"tools": [{"type": "mcp_toolset", "mcp_server_name": "image-probe"}]
}'
```

**3.** Observe the `mcp_tool_result` block. The image block is gone, replaced by text.

### Result — the drop is selective

The same tool result carries a token in text and a token in pixels:

| token | carried in | reached the model |
|---|---|---|
| `TEXTPATH-CEDAR-08` | `text` block | yes |
| `PURPLE-RHINO-42` | `image` block (pixels only) | **no** |

The model reported the text-token correctly and `NOT VISIBLE` for the image-token. So the tool call,
the transport, and the server are all working; only the image block is removed.

### Control — the same PNG works outside MCP

Same file, same public URL, same model, passed as an ordinary message content block instead of through
MCP:

```json
{"type": "image", "source": {"type": "url", "url": "https:///probe.png"}}
```

The model answered `PURPLE-RHINO-42`, and my server logged the inbound GET.

This isolates the connector as the point of failure: it is not the file, not the model's vision, and not
the API's ability to fetch a URL.

### Also checked

- **`resource_link`** (the MCP spec's by-reference form for exactly this case) is rejected by block type
with `[MCP tool returned content of type 'ResourceLink' which is not currently supported by the
Anthropic API]`. My server logged no fetch attempt for the URI.
- A URL placed in a plain `text` block passes through as text but is never fetched.
- Reproduced identically on `claude-opus-5`, `claude-sonnet-5`, `claude-haiku-4-5-20251001` and
`claude-fable-5` — same placeholder string every time, so this is not model-specific.

Taken together there is no shape an MCP server can return that delivers an image to the model over the
hosted connector — neither inline nor by reference.

## Error messages or logs

```shell
# mcp_tool_result content, is_error=false
[
{"type": "text", "text": "text-token=TEXTPATH-CEDAR-08. The image block that follows carries a different image-token; read it from the pixels."},
{"type": "text", "text": "[MCP tool returned an image (format: image/png) but the Anthropic API doesn't currently support image responses]"}
]

# resource_link variant
{"type": "text", "text": "[MCP tool returned content of type 'ResourceLink' which is not currently supported by the Anthropic API]"}
```

Across the whole API response: `"type": "image"` × 0, `base64` × 0.

## Additional context

**Requests, in priority order:**

1. **Document it.** Whatever the outcome on the behavior, the MCP connector Limitations section should
state that `image` content blocks in tool results are not forwarded to the model, and that
`resource_link` is not supported. Right now this is discoverable only by building a probe.
2. **Make it loud.** `is_error: false` on a result whose payload was discarded is a silent failure. An
error, or at minimum a documented signal the caller can detect, would let callers fall back instead
of shipping a broken path.
3. **Forward the image**, if the intent is that MCP tool results can carry them.

**Environment**

- Raw HTTPS to `api.anthropic.com/v1/messages` — no Anthropic SDK (`urllib` only), so no client-side
serialization is involved
- Beta header `mcp-client-2025-11-20`
- MCP server: hand-written Python `http.server`, streamable HTTP, no auth, no MCP SDK
- Observed 2026-09-01

**On whether this is a regression:** a commenter on #643 reports same-turn MCP image vision working for
roughly four months before mid-July 2026 on the chat surface. I cannot confirm that for the API
connector — my testing only establishes current behavior.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.