feat(ai-proxy): support OpenAI Chat Completions clients with native Anthropic Messages API upstreams
- Dominant language
- Lua
- Stars
- 17.1k
- Forks
- 2.9k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 63
Description
### Description
ai-proxy/ai-proxy-multi can translate Anthropic-format clients → OpenAI-compatible upstreams (via the anthropic-messages-to-openai-chat converter), but not the reverse. There is currently no way to accept OpenAI Chat Completions requests and forward them to an upstream that only speaks the native Anthropic Messages API (/v1/messages).
I'm trying to proxy an Open WebUI message to Claude on Azure AI Foundry, whose [API surface](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/claude-models#api-surface) is exclusively the native Anthropic Messages API (https://.services.ai.azure.com/anthropic/v1/messages), exposing no OpenAI-compatible /chat/completions endpoint.
The APISIX anthropic provider's openai-chat capability assumes the upstream offers Anthropic's own OpenAI-compatible endpoint (api.anthropic.com/v1/chat/completions). When pointed at a Messages-only upstream via override.endpoint, the OpenAI body is passed through unchanged and rejected by the upstream.
### Steps to reproduce
Route with ai-proxy-multi, provider: anthropic, and override.endpoint set to an Azure Foundry Claude deployment (.../anthropic/v1/messages). Send a standard OpenAI Chat Completions request:
```
curl "$GATEWAY/v1/chat/completions" -X POST \
-H "Content-Type: application/json" \
-d '{
"model": "claude-opus-4-5",
"messages": [
{ "role": "system", "content": "You are a mathematician" },
{ "role": "user", "content": "What is 1+1?" }
]
}'
```
Actual behavior
The request is forwarded to the Messages API verbatim (still OpenAI-shaped), and the upstream rejects it:
{"type":"error","error":{"type":"invalid_request_error",
"message":"messages.0: use the top-level 'system' parameter for the initial system prompt"}}
The OpenAI convention of a system role inside messages[] is incompatible with the Messages API, which requires a top-level system parameter (among other format differences: max_tokens required, content blocks, input_tokens/output_tokens usage fields, etc.).
Expected behavior
ai-proxy/ai-proxy-multi should be able to accept an OpenAI Chat Completions request from the client and deliver it to a native Anthropic Messages API upstream in the correct format (request and response, streaming and non-streaming), so that OpenAI-format clients can transparently use Anthropic-only backends such as Azure AI Foundry.
Environment
APISIX version: v3 (Helm chart 2.14.1)
Plugin: ai-proxy-multi (also applies to ai-proxy), provider: anthropic
Contributor guide
Research direction
Start at the ai-proxy-multi Anthropic provider and its override.endpoint handling, comparing it with the existing anthropic-messages-to-openai-chat converter. Use the documented /v1/chat/completions request and the native /v1/messages endpoint as the first reproduction cases. Done means compatible request and response translation for streaming and non-streaming clients.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, lua
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100