posit-dev / posit-dev/chatlas

Anthropic provider drops empty assistant turns, breaking role alternation

Open Beginner friendly
#416 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ai-triage:done Priority: Medium
Dominant language
Python
Stars
176
Forks
28
Avg merge
18h 42m
Merged PRs (30d)
16

Description

Summary

AnthropicProvider._as_message_params (chatlas/_provider_anthropic.py:884-887) drops assistant turns that have zero content blocks instead of sending a placeholder:

# Drop empty assistant turns to avoid an API error
# (all messages must have non-empty content)
if turn.role == "assistant" and len(content) == 0:
    continue

If a turn is dropped this way, it can produce two consecutive user-role messages in the outgoing request, which violates Anthropic's strict requirement that messages alternate user/assistant. This is the same class of bug ellmer just fixed (tidyverse/ellmer#1099, tidyverse/ellmer#1100; see also tidyverse/ellmer#711, tidyverse/ellmer#1070): ellmer now sends a "[empty string]" placeholder instead of dropping the turn, because dropping it both risks an alternation error and "confuses the model."

Note: chatlas already normalizes empty/whitespace text to "[empty string]" at the ContentText level (chatlas/_content.py:288-292), which covers the common case where a model returns an empty or whitespace-only text block (confirmed via the existing Databricks empty-response VCR cassette, tests/_vcr/test_provider_databricks/test_databricks_empty_response.yaml). The bug here is narrower: it's specifically about an AssistantTurn with a genuinely empty content list (contents=[]), which never goes through ContentText's normalization and instead hits the continue branch above.

Repro

from chatlas._provider_anthropic import AnthropicProvider
from chatlas._turn import AssistantTurn, UserTurn

provider = AnthropicProvider(
    name="Anthropic",
    model="claude-3-5-haiku-latest",
    api_key="dummy-key-not-used",
)

turns = [
    UserTurn("Respond with only two blank lines"),
    AssistantTurn(contents=[], finish_reason="success"),
    UserTurn("What's 1+1? Just give me the number"),
]

messages = provider._as_message_params(turns)
print([m["role"] for m in messages])
# ['user', 'user']  <-- two consecutive user messages; Anthropic's API would reject this

Open question for triage: it's not yet confirmed whether Anthropic's real response-parsing path can ever actually produce an AssistantTurn with contents=[] (as opposed to always synthesizing at least one, possibly-empty, ContentText block first, which would already get the "[empty string]" treatment upstream and never reach this branch). Regardless, the message-construction logic itself is demonstrably unsafe if such a turn is ever constructed (e.g. programmatically, or via a future code path), so the fix is worth making independent of how often it's hit today.

Expected fix

Mirror ellmer's fix: replace the continue with a placeholder content block, e.g. [{"type": "text", "text": "[empty string]"}], instead of dropping the turn.

References

  • ellmer PRs: tidyverse/ellmer#1099, tidyverse/ellmer#1100 (issues: tidyverse/ellmer#711, tidyverse/ellmer#1070)
  • chatlas: chatlas/_provider_anthropic.py:884-887, chatlas/_content.py:288-292

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in chatlas/_provider_anthropic.py at AnthropicProvider._as_message_params (884-887), then run the provided AssistantTurn reproduction to inspect the outgoing roles. Check the related normalization in chatlas/_content.py and confirm the result preserves user/assistant alternation with placeholder content for an empty assistant turn.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.