cloudwego / cloudwego/eino-ext

components/model/claude: translate file/PDF content parts to Anthropic document blocks

Open
#918 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
811
Forks
368
Avg merge
16h 22m
Merged PRs (30d)
13

Description

## Summary

The Claude chat model (`components/model/claude`) does not translate a `schema.ChatMessagePartTypeFileURL` content part into the Anthropic Messages API `document` block. Any message (or tool result) carrying a PDF file part falls into the `default` branch of the content-conversion switch and fails with:

```
anthropic message type not supported: file
```

This blocks the multimodal `read_file` path: the `adk/middlewares/filesystem` middleware with `UseMultiModalRead: true` (over a `MultiModalReader` backend such as `adk/backend/local`) returns a PDF as a `FileContentPartTypePDF` part, which the `schema` layer converts to `MessageInputPart{Type: ChatMessagePartTypeFileURL, File}`. When that part reaches the Claude model, conversion errors out — so a PDF read cannot be delivered to the model as a native document.

## Why it matters

The Anthropic Messages API supports a first-class `document` block for PDFs (base64 or URL), letting the model read the PDF natively (text + vision handled server-side), avoiding client-side rasterization into images. The `anthropic-sdk-go` currently used by this module already exposes `DocumentBlockParam`, `Base64PDFSourceParam`, `URLPDFSourceParam`, and — importantly — `ToolResultBlockParamContentUnion.OfDocument`, so a document can be carried both in a user message and in a tool result. Only the translation layer in `components/model/claude/claude.go` is missing the `case`.

## Proposal

Add a `case schema.ChatMessagePartTypeFileURL` to the content-conversion switches in `convSchemaMessage` (user-input multi content) and `convToolMultiContent` (tool result), mapping a PDF file part to an Anthropic `document` block. It mirrors the existing image handling exactly (URL vs raw base64; MIMEType required for base64; reject `data:` prefix), with an added `application/pdf` guard because the base64 document source is PDF-only.

## Scope

- `components/model/claude/claude.go`: two new `case` branches + two small helpers.
- Tests mirroring `Test_convSchemaMessage_MultiContent` for the document path (base64, URL, `data:`-prefix error, missing-MIME error, non-PDF-MIME error, missing-source error; plus the tool-result path).
- No new dependency.

## Verified

- The Anthropic Messages endpoint accepts a `document` block inside a tool result and in a user message (tested end-to-end against a real PDF).
- `ToolResultBlockParamContentUnion` has `OfDocument` in the SDK version this module pins, so no SDK bump is required.

I have a change + tests ready on a branch and am happy to open a PR. Does this direction look acceptable?

Contributor guide

Open the contributing guide

Research direction

Start in components/model/claude/claude.go by reading convSchemaMessage and convToolMultiContent, then inspect the existing image conversion and the Anthropic SDK document types named in the issue. Add coverage mirroring Test_convSchemaMessage_MultiContent for base64 and URL PDFs, validation errors, and tool results. Done means PDF file parts become document blocks in both message paths without breaking existing conversions.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.