posit-dev / posit-dev/chatlas

`ChatBedrock()` mantle APIs send URL content sources, which Bedrock rejects

Open
#410 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ai-triage:done Priority: Medium
Dominant language
Python
Stars
176
Forks
28
Avg merge
18h 42m
Merged PRs (30d)
16

Description

When porting #345 to ellmer in https://github.com/tidyverse/ellmer/pull/1101/, I found an edge case; 🤖 summary below:

content_pdf_url() (and content_document_url() on the responses API) now pass the URL to the provider instead of sending bytes. BedrockMessagesProvider and BedrockResponsesProvider inherit that from AnthropicProvider and OpenAIProvider, but bedrock-mantle can't fetch external URLs, so both 400. Local files work fine on both endpoints.

import chatlas as ctl

pdf = ctl.content_pdf_url("https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf")
doc = ctl.content_document_url("https://raw.githubusercontent.com/tidyverse/ellmer/main/README.md")

ctl.ChatBedrock(model="anthropic.claude-sonnet-5", api="messages").chat("What is this?", pdf)
#> 400: URL content sources are not yet supported for this model

ctl.ChatBedrock(model="openai.gpt-5.4").chat("What is this?", pdf)
#> 400: Unsupported file type: 'unknown'. Supported: PDF, DOCX, XLSX, CSV, TXT, MD, HTML, ...

ctl.ChatBedrock(model="openai.gpt-5.4").chat("Summarise this.", doc)
#> 400: same

content_image_url() fails the same way on both endpoints ("must be data: or s3://" on responses), though that predates #345.

The fix in ellmer overrides content serialisation on the two mantle classes to drop the URL and fall back to the bytes it already has. Since ContentPDF.data is optional in chatlas, the fallback here would need to go through ensure_bytes() to actually download.

Tested against chatlas main (0.22.1.dev5), anthropic 1.2.0, openai 3.6.0.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the content serialization paths in BedrockMessagesProvider and BedrockResponsesProvider, comparing them with their inherited AnthropicProvider and OpenAIProvider behavior. Follow ensure_bytes() for URL-backed content and verify that PDF and document URLs are sent as bytes rather than URLs; the existing local-file behavior should remain working.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.