dapr / dapr/components-contrib

conversation/anthropic: pinned langchaingo blocks Claude models newer than the 4.6 generation

Open
#4,533 3 comments 0 reactions 1 assignee Claimed by @vaishaldsouza View on GitHub
good first issue
Dominant language
Go
Stars
602
Forks
580
Avg merge
4d 9h
Merged PRs (30d)
6

Description

## Summary

The `conversation.anthropic` component cannot use any Claude model newer than the 4.6 generation. Two separate problems in the pinned `github.com/tmc/langchaingo` are responsible, and bumping the dependency currently makes things worse rather than better.

Pinned version: `v0.1.15-0.20251029190607-e35755df7084` (2025-10-29).

## Problem 1: `temperature` is always serialized

`llms/anthropic/internal/anthropicclient/messages.go`:

```go
type messagePayload struct {
Model string `json:"model"`
Messages []ChatMessage `json:"messages"`
System string `json:"system,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
StopWords []string `json:"stop_sequences,omitempty"`
Stream bool `json:"stream,omitempty"`
Temperature float64 `json:"temperature"` // <-- no omitempty
Tools []Tool `json:"tools,omitempty"`
TopP float64 `json:"top_p,omitempty"`
...
}
```

`Temperature` is the only optional field in the struct without `omitempty`, so `"temperature": 0` goes out on every Anthropic request even when the caller never set one. `langchaingokit.getOptionsFromRequest` correctly skips `llms.WithTemperature` when `Request.Temperature == 0`, but that has no effect — the zero value is serialized regardless.

Claude models from the 4.7 generation onward removed sampling parameters, so every request fails:

```
400: `temperature` is deprecated for this model.
```

Affects Claude Sonnet 5, Opus 5, Opus 4.8, and Opus 4.7.

## Problem 2: bumping langchaingo removes tool_choice

langchaingo main (`v0.1.15-0.20260111165956-8fea3de63675`, 2026-01-11) deleted Anthropic `tool_choice` support entirely:

- `ToolChoice` is gone from `messagePayload`
- the `tool_choice` JSON key is gone
- `grep -rn "ToolChoice" llms/anthropic/` returns nothing — `opts.ToolChoice` is never read
- `anthropicllm.go` now reads only `opts.Tools`, `opts.Temperature`, `opts.TopP`

`Temperature` still has no `omitempty` on that commit, so bumping loses tool choice **and** keeps the 400.

## Impact

- The component is capped at the 4.6-generation models. `DefaultAnthropicModel` had to be set to `claude-sonnet-4-6` rather than tracking the current Claude release.
- Adaptive thinking, the `effort` parameter, and the 4.7+ capability improvements are unreachable.
- Any `tool_choice` fix in `langchaingokit` depends on langchaingo passing the value through verbatim. That passthrough no longer exists upstream, so a future bump would silently stop sending `tool_choice` — the request still succeeds, the model just behaves as `auto`.

## Options

**A. Fix upstream, then bump.** Two langchaingo PRs: add `,omitempty` to `Temperature`, and restore Anthropic `tool_choice` (ideally emitting the object form the Messages API requires). Smallest diff here, but depends on an external merge. If upstream adds its own object translation, the translation in `langchaingokit` has to be removed in the same change or the two will conflict.

**B. `replace` directive to a patched fork.** Unblocks immediately, but pins the project to a fork and adds an ongoing rebase burden. Not appropriate for this repo.

**C. Use `anthropic-sdk-go` directly for the Anthropic component.** Removes this entire class of bug at once — `temperature`, `tool_choice`, extended thinking, and the prompt-cache-retention workaround documented in `conversation/langchaingokit/model.go`.

Cost: `conversation.Request`/`Response` are built on langchaingo types (`llms.MessageContent`, `llms.Tool`, `llms.ToolCall`), so the public surface stays as-is and an adapter converts at the component boundary. Larger change, contained to one component, and the only route to the current Claude models.

## Reproduction

```bash
export ANTHROPIC_API_KEY=...
export ANTHROPIC_MODEL=claude-sonnet-5
go test -tags conftests ./tests/conformance -run 'TestConversationConformance/anthropic' -v
```

Every subtest fails with `400: \`temperature\` is deprecated for this model.` Re-running with `ANTHROPIC_MODEL=claude-sonnet-4-6` passes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.