anthropics / anthropics/claude-ai-mcp

[claude.ai / MCP] Image content blocks from tool results not rendered inline in assistant response

Aperta
#238 6 commenti 18 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Nessun dato sulla lingua
Stelle
471
Fork
77
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### Problem Statement

When an MCP tool returns an `image` content block (base64-encoded PNG), Claude can see and reason about the image — but it is never rendered inline in the final assistant response visible to the user.

**Environment:**
- Platform: claude.ai (web)
- - MCP server: remote (SSE), connected via Claude.ai Settings → Connections
- - - Tool type: chart rendering tool returning base64 PNG + URL
**What our tool returns:**
```json
[
{
"type": "image",
"data": "",
"mimeType": "image/png"
},
{
"type": "text",
"text": "Chart rendered: https://example.com/chart.png"
}
]
```

**Observed behavior:**
1. Claude **can see the image** — it appears in the thinking/reasoning block and Claude correctly describes its contents
2. 2. The image appears inside the collapsed "tool use" accordion (which most users never expand)
3. 3. The image is **never rendered inline** in the final assistant response
**Expected behavior:**
When Claude references a tool result that contains an `image` block, that image should render inline in the assistant response — similar to how user-uploaded images are displayed.

**Why this matters:**
MCP is designed as the standard for extending Claude with external tools. Visual tools (charts, diagrams, screenshots) are a major category. If image results can't surface to the user without manual accordion expansion, it breaks the core value proposition of visual MCP tools.

### Proposed Solution

Please support inline rendering of `image` content blocks from MCP tool results in the assistant response — either:
1. **Automatically** when Claude references the image in its response, or
2. 2. **Via an explicit opt-in mechanism** for tool developers (e.g., a flag in the tool result or server manifest)
This would work similarly to how user-uploaded images are already rendered inline in the chat UI.

### Alternatives Considered

All workarounds are suboptimal:

1. **Returning a URL in text** → external images not rendered inline by claude.ai
2. 2. **Building an HTML artifact wrapper** → requires extra roundtrip, bad UX; also blocked by the widget sandbox CSP (all external image domains are blocked)
3. 3. **Instructing users to open the accordion** → not discoverable by end users
The Anthropic documentation explicitly recommends the base64 `image` content block approach for MCP tools, but the UI does not render these images inline, creating a contradiction.

### Area

Tool Discovery / Invocation

### Priority

High - Major improvement needed

### Additional Context

This issue was originally filed in anthropics/anthropic-sdk-python#1329 but was redirected here as the correct repository.

Multiple developers have confirmed this issue:
- Chart rendering tools returning base64 PNG images
- - Photography validation tools returning base64 image blocks
- - - The Anthropic documentation explicitly recommends the base64 `image` content block approach for MCP tools, but the claude.ai UI does not render these images inline
This creates a direct contradiction between the documented best practice and the actual behavior of claude.ai.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.