api7 / api7/aisix

feat(messages): forward Anthropic response headers (`anthropic-ratelimit-*`, `request-id`) on /v1/messages

Open
#318 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement priority-normal
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Background

Surfaced while auditing the architecture deep-dive doc for /v1/messages
(PR #317). The doc
initially claimed the byte-passthrough path forwards Anthropic-specific
response headers verbatim. The independent audit caught that the
implementation does not, and the doc has been corrected to match reality.
This issue tracks the underlying product gap.

What the code does today

/v1/messages against an Anthropic upstream:

  • Streaming path (crates/aisix-proxy/src/messages.rs:283-301)
    copies only content-type from the upstream response. It also
    injects cache-control: no-cache and x-aisix-request-id. No
    other upstream header is forwarded.
  • Non-streaming path (messages.rs:344-348)
    rebuilds the response via Json(json_body).into_response(), which
    produces a fresh response with content-type: application/json and
    no upstream headers attached.

Why this matters

Anthropic emits several headers the SDK and operators rely on:

  • anthropic-ratelimit-requests-limit /
    anthropic-ratelimit-requests-remaining /
    anthropic-ratelimit-requests-reset
  • anthropic-ratelimit-tokens-limit /
    anthropic-ratelimit-tokens-remaining /
    anthropic-ratelimit-tokens-reset
  • request-id — Anthropic's per-request identifier; useful for
    cross-correlating gateway logs with Anthropic support tickets

The Anthropic Python SDK reads anthropic-ratelimit-* for
back-pressure; passing them through is the entire point of running
a byte-passthrough path against an Anthropic upstream.

Proposed fix

Add a small allowlist of upstream headers to forward on the
streaming passthrough path (in addition to content-type):

const FORWARDED_ANTHROPIC_HEADERS: &[&str] = &[
    "anthropic-ratelimit-requests-limit",
    "anthropic-ratelimit-requests-remaining",
    "anthropic-ratelimit-requests-reset",
    "anthropic-ratelimit-tokens-limit",
    "anthropic-ratelimit-tokens-remaining",
    "anthropic-ratelimit-tokens-reset",
    "request-id",
];

for name in FORWARDED_ANTHROPIC_HEADERS {
    if let Some(v) = headers.get(*name) {
        if let Ok(hv) = HeaderValue::from_bytes(v.as_bytes()) {
            response.headers_mut().insert(
                HeaderName::from_static(name),
                hv,
            );
        }
    }
}

Mirror the same allowlist on the non-streaming path: switch from
Json(...).into_response() to building the response manually
(Response::builder() with status / headers / body), so we can
preserve the upstream's allowlisted headers.

Allowlist (rather than passthrough-all) so we don't forward
upstream Set-Cookie, internal tracing IDs, or other headers that
shouldn't cross the customer boundary.

Acceptance criteria

  • Allowlisted Anthropic headers reach the client on the
    streaming /v1/messages path
  • Allowlisted Anthropic headers reach the client on the
    non-streaming /v1/messages path
  • E2E test asserts each header name and value round-trips
    unchanged on both paths (using a recorded fixture against the
    real Anthropic upstream, or a controlled mock that mirrors the
    Anthropic header set)

Out of scope

  • Same forwarding policy on the cross-provider synthesis path
    (OpenAI / Gemini / DeepSeek don't emit anthropic-ratelimit-*,
    and the proxy can't synthesise it without re-implementing the
    rate-limit math against the upstream's own counters)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/aisix-proxy/src/messages.rs, reviewing the streaming response handling at lines 283-301 and non-streaming handling at lines 344-348. Use a controlled mock or recorded fixture with the listed Anthropic headers, then add E2E coverage for both paths. Done means every allowlisted header reaches the client with its value unchanged on streaming and non-streaming /v1/messages responses.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.