anthropics / anthropics/claude-ai-mcp

[claude.ai / MCP] Image content blocks from tool results not rendered inline in assistant response

Open
#238 6 comments 18 reactions 0 assignees View on GitHub
enhancement
Dominant language
No language data
Stars
471
Forks
76
PR merge metrics
No merged PRs in 30d

Description

### Problem Statement

When an MCP tool returns an `image` content block (base64-encoded PNG), Claude can see and reason about the image — but it is never rendered inline in the final assistant response visible to the user.

**Environment:**
- Platform: claude.ai (web)
- - MCP server: remote (SSE), connected via Claude.ai Settings → Connections
- - - Tool type: chart rendering tool returning base64 PNG + URL
**What our tool returns:**
```json
[
{
"type": "image",
"data": "",
"mimeType": "image/png"
},
{
"type": "text",
"text": "Chart rendered: https://example.com/chart.png"
}
]
```

**Observed behavior:**
1. Claude **can see the image** — it appears in the thinking/reasoning block and Claude correctly describes its contents
2. 2. The image appears inside the collapsed "tool use" accordion (which most users never expand)
3. 3. The image is **never rendered inline** in the final assistant response
**Expected behavior:**
When Claude references a tool result that contains an `image` block, that image should render inline in the assistant response — similar to how user-uploaded images are displayed.

**Why this matters:**
MCP is designed as the standard for extending Claude with external tools. Visual tools (charts, diagrams, screenshots) are a major category. If image results can't surface to the user without manual accordion expansion, it breaks the core value proposition of visual MCP tools.

### Proposed Solution

Please support inline rendering of `image` content blocks from MCP tool results in the assistant response — either:
1. **Automatically** when Claude references the image in its response, or
2. 2. **Via an explicit opt-in mechanism** for tool developers (e.g., a flag in the tool result or server manifest)
This would work similarly to how user-uploaded images are already rendered inline in the chat UI.

### Alternatives Considered

All workarounds are suboptimal:

1. **Returning a URL in text** → external images not rendered inline by claude.ai
2. 2. **Building an HTML artifact wrapper** → requires extra roundtrip, bad UX; also blocked by the widget sandbox CSP (all external image domains are blocked)
3. 3. **Instructing users to open the accordion** → not discoverable by end users
The Anthropic documentation explicitly recommends the base64 `image` content block approach for MCP tools, but the UI does not render these images inline, creating a contradiction.

### Area

Tool Discovery / Invocation

### Priority

High - Major improvement needed

### Additional Context

This issue was originally filed in anthropics/anthropic-sdk-python#1329 but was redirected here as the correct repository.

Multiple developers have confirmed this issue:
- Chart rendering tools returning base64 PNG images
- - Photography validation tools returning base64 image blocks
- - - The Anthropic documentation explicitly recommends the base64 `image` content block approach for MCP tools, but the claude.ai UI does not render these images inline
This creates a direct contradiction between the documented best practice and the actual behavior of claude.ai.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.