Preserve HTTP overload status and Retry-After through streaming shim

Overview

Impact: High under overload. Priority: P1 (provisional pending real-client retry qualification). Estimated effort: Medium.

The Anthropic shim converts an upstream HTTP429 with Retry-After into HTTP200 plus an SSE error and drops the retry delay. A streaming Claude client cannot receive the server's advertised backoff via normal HTTP headers. Nonstream requests preserve both, so nonstream rate-limit tests miss this.

Current state and reproduction

Audited main 093cd44806ec64b2c79b42b557b4856259d2ea58:

• Chunked response starts before upstream request/status.
• Upstream failure becomes SSE error only.
• Nonstream correctly copies429 and Retry-After.

Hermetic loopback test compiled unchanged messages.cpp, translate.cpp, and loopback_auth.cpp with the repo-pinned cpp-httplib0.46.1/nlohmann3.11.3; only output/state-directory functions were stubbed. No credentials or real inference API used.

1. Mock upstream /v1/chat/completions responds status429, Retry-After: 7, body {"error":{"message":"Too many parallel requests"}}.
2. Start real shim against mock.
3. Send authenticated local /v1/messages request {"model":"glm-5.3-flash","stream":true,"max_tokens":128,"messages":[{"role":"user","content":"hi"}]}.
4. Observe actual output:

429 stream downstream_status=200 retry_after= body=event: error
data: {"error":{"message":"Too many parallel requests","type":"rate_limit_error"},"type":"error"}

The loss is proven; repeated immediate retries by a particular Claude SDK version are NOT yet claimed. Public API overload behavior still needs qualification later.

Proposed solution and implementation

• Read upstream headers before committing the downstream streaming status/headers; preserve status and Retry-After for a pre-stream refusal.
• Keep typed SSE errors for failures after output has started; document retry semantics separately for that case.
• Add loopback HTTP tests for429,503 and successful streaming; retain nonstream regression coverage.
• Verify the wrapped client's retry delay using a fake upstream that counts arrival timestamps.

Success criteria

Pre-stream overload reaches the client with actionable retry semantics including the upstream delay; no immediate retry loop; errors after partial output are never replayed as fresh inference silently. Actual production backoff test is deferred until API access is supplied.

Related: #80, #81, #82 (same transport code; coordinate changes). This issue adds no release-blocker label; release triage is deferred.

Implementation and merge status

Implemented in PR #103 at af0a4823ee6e69414da2413cc2b525b524a91eae; final independent published-head review found no unresolved blocking findings. See the final review comment for exact tests and remaining platform/live acceptance. Keep this issue open.

Merge owner: @Siddhesh2377. Sequence: #82 → #103 → #104, ultimately main. Rebase and retarget dependent PRs after prerequisites merge, then rerun checks. Nothing has been deployed by this implementation work.