[Bug] Claude Code requests fail on Azure Opus when thinking parameters are not normalized per target

Environment

• Repo / commit: NVIDIA-NeMo/Switchyard @ 3acf3d86f10fead9f434b858dbbcf01a660b8db0
• Switchyard: 0.2.0
• OS: macOS 26.5.2
• Python: 3.13.5
• Claude Code: 2.1.220
• Key packages: anthropic==0.99.0, litellm==1.85.0
• Upstream: NVIDIA Inference Hub (https://inference-api.nvidia.com/v1)
• Models:
  • aws/anthropic/bedrock-claude-opus-5
  • azure/anthropic/claude-opus-5

Description

When Switchyard routes Claude Code traffic between the Bedrock and Azure deployments of Claude Opus 5, requests selected for Bedrock succeed while requests selected for Azure fail with HTTP 400.

Claude Code sends Anthropic thinking parameters using thinking.type=enabled. The Azure deployment requires thinking.type=adaptive and output_config.effort. Switchyard currently does not normalize this provider/model-specific difference after selecting the target, so an otherwise valid routing policy becomes partially unavailable.

The final launcher summary compounds the confusion because its per-model req value reports successful calls only, while failed attempts appear only in the aggregate error count. This can make it look as though Bedrock received all traffic even when Azure was selected and failed.

Steps to Reproduce

1. Create /tmp/opus-deployment-random.yaml:

   defaults:
     api_key: ${NVIDIA_INFERENCE_HUB_API_KEY}
     base_url: ${NVIDIA_INFERENCE_HUB_API_ROOT}
     format: openai
   
   routes:
     demo/opus-deployment-random:
       type: random_routing
       strong_probability: 0.5
       fallback_target_on_evict: bedrock
       strong:
         id: bedrock
         model: aws/anthropic/bedrock-claude-opus-5
       weak:
         id: azure
         model: azure/anthropic/claude-opus-5
   

2. Start Switchyard:

   source .venv/bin/activate
   switchyard serve --routing-profiles /tmp/opus-deployment-random.yaml --port 4000
   

3. In another terminal, route Claude Code through the proxy:

   ANTHROPIC_BASE_URL=http://127.0.0.1:4000 \
   ANTHROPIC_AUTH_TOKEN=switchyard \
   ANTHROPIC_API_KEY="" \
   ANTHROPIC_MODEL=demo/opus-deployment-random \
   ANTHROPIC_SMALL_FAST_MODEL=demo/opus-deployment-random \
   claude
   

4. Send several normal requests, such as Reply with only OK. For a deterministic comparison, temporarily set strong_probability: 1.0 to exercise Bedrock, then 0.0 to exercise Azure.

Actual Result

The Bedrock target succeeds. The Azure target returns:

400 litellm.BadRequestError: Azure_aiException -
{"type":"error","error":{"type":"invalid_request_error",
"message":"\"thinking.type.enabled\" is not supported for this model.
Use \"thinking.type.adaptive\" and \"output_config.effort\" to control
thinking behavior."}}

A representative final summary was:

requests : 20  (11 errors)
models:
  bedrock-claude-opus-5  9 req
  claude-opus-5          0 req

A minimal Azure request without Claude Code's thinking configuration succeeds, confirming that the deployment, model ID, credentials, and endpoint are valid.

Expected Result

Switchyard should normalize request parameters for the selected target so the same Claude Code request succeeds on both deployments.

For Azure Opus, enabled thinking should be represented as adaptive thinking, with supported effort configuration and without a fixed thinking budget. The session summary should also display per-model failures so failed selections are distinguishable from models that were never selected.

Suggested Fix

Add target-aware request normalization at the outbound backend boundary, after routing has selected the target:

• Rewrite unsupported thinking.type=enabled to thinking.type=adaptive for affected Azure Opus targets.
• Remove incompatible fixed-budget fields.
• Preserve supported effort through output_config.effort.
• Add tests that send the same Claude Code request through the Bedrock and Azure targets and assert the provider-appropriate outbound payload.
• Include each model's error count in the launcher session summary.

Target-level extra_body alone cannot reliably implement this because its current shallow merge is caller-wins and therefore cannot replace an inbound thinking field.

Start at the outbound backend boundary after routing selects a target, then inspect the launcher session summary described in the issue. Reproduce with the provided routing profile and Claude Code commands, comparing Bedrock and Azure payloads. Done means both targets accept the request with provider-appropriate thinking parameters and the summary reports per-model failures.