fix: AnthropicLLM uses deprecated completions API, broken for Claude 3+ and drops full conversation history
- Dominant language
- Python
- Stars
- 18.3k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 29
Description
## Bug Description
`application/llm/anthropic.py` uses the legacy Claude 2-era text completions API:
```python
# lines 47-52
completion = self.anthropic.completions.create(
model=model,
max_tokens_to_sample=max_tokens,
prompt=f"{self.HUMAN_PROMPT} {prompt}{self.AI_PROMPT}",
)
```
This API is not supported by any Claude 3/3.5/4 model. Additionally both `_raw_gen` and `_raw_gen_stream` only extract `messages[0]` and `messages[-1]`, silently discarding the entire multi-turn conversation history between them. All other LLM providers pass the full `messages` list.
## Expected Behavior
`AnthropicLLM` should use `client.messages.create()` / `client.messages.stream()` (the modern Messages API), pass the full conversation history, and correctly handle system messages via the `system=` parameter.
## Proposed Fix
- Remove deprecated `HUMAN_PROMPT`/`AI_PROMPT` imports and instance variables
- Rewrite `_raw_gen` to use `anthropic.messages.create()`, return `response.content[0].text`
- Rewrite `_raw_gen_stream` to use `anthropic.messages.stream()`, yield text deltas
- Add `_split_messages()` helper to separate system prompt from conversation turns
Contributor guide
Assessment
This issue has not been assessed yet.