apache / apache/apisix

feat(ai-proxy): support OpenAI Chat Completions clients with native Anthropic Messages API upstreams

Open
#13,566 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Lua
Stars
17.1k
Forks
2.9k
Avg merge
3d 16h
Merged PRs (30d)
63

Description

### Description

ai-proxy/ai-proxy-multi can translate Anthropic-format clients → OpenAI-compatible upstreams (via the anthropic-messages-to-openai-chat converter), but not the reverse. There is currently no way to accept OpenAI Chat Completions requests and forward them to an upstream that only speaks the native Anthropic Messages API (/v1/messages).

I'm trying to proxy an Open WebUI message to Claude on Azure AI Foundry, whose [API surface](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/claude-models#api-surface) is exclusively the native Anthropic Messages API (https://.services.ai.azure.com/anthropic/v1/messages), exposing no OpenAI-compatible /chat/completions endpoint.

The APISIX anthropic provider's openai-chat capability assumes the upstream offers Anthropic's own OpenAI-compatible endpoint (api.anthropic.com/v1/chat/completions). When pointed at a Messages-only upstream via override.endpoint, the OpenAI body is passed through unchanged and rejected by the upstream.

### Steps to reproduce
Route with ai-proxy-multi, provider: anthropic, and override.endpoint set to an Azure Foundry Claude deployment (.../anthropic/v1/messages). Send a standard OpenAI Chat Completions request:
```
curl "$GATEWAY/v1/chat/completions" -X POST \
-H "Content-Type: application/json" \
-d '{
"model": "claude-opus-4-5",
"messages": [
{ "role": "system", "content": "You are a mathematician" },
{ "role": "user", "content": "What is 1+1?" }
]
}'
```

Actual behavior
The request is forwarded to the Messages API verbatim (still OpenAI-shaped), and the upstream rejects it:

{"type":"error","error":{"type":"invalid_request_error",
"message":"messages.0: use the top-level 'system' parameter for the initial system prompt"}}
The OpenAI convention of a system role inside messages[] is incompatible with the Messages API, which requires a top-level system parameter (among other format differences: max_tokens required, content blocks, input_tokens/output_tokens usage fields, etc.).

Expected behavior
ai-proxy/ai-proxy-multi should be able to accept an OpenAI Chat Completions request from the client and deliver it to a native Anthropic Messages API upstream in the correct format (request and response, streaming and non-streaming), so that OpenAI-format clients can transparently use Anthropic-only backends such as Azure AI Foundry.

Environment
APISIX version: v3 (Helm chart 2.14.1)
Plugin: ai-proxy-multi (also applies to ai-proxy), provider: anthropic

Contributor guide

Open the contributing guide

Research direction

Start at the ai-proxy-multi Anthropic provider and its override.endpoint handling, comparing it with the existing anthropic-messages-to-openai-chat converter. Use the documented /v1/chat/completions request and the native /v1/messages endpoint as the first reproduction cases. Done means compatible request and response translation for streaming and non-streaming clients.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, lua
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.