authbridge-proxy: early flush of leading thinking block turns a slow tool call into a client stream-idle timeout

Summary

When authbridge-proxy is on the path as a forward proxy with TLS bridge enabled, a long-running
tool call from an Anthropic-style streaming endpoint is delivered to the client in a shape that
trips Claude Code's 300 s stream-idle watchdog. The same request bypassing AuthBridge completes
normally.

The upstream gateway (LiteLLM fronting AWS Bedrock) withholds a tool_use content block until it is
fully generated — this happens on both paths and is not an AuthBridge bug. The AuthBridge-specific
behaviour is that it flushes the leading thinking block to the client immediately (~5–16 s),
while the direct path holds everything and delivers the response near-atomically at the end.

Consequence: with AuthBridge the entire tool-call generation becomes a single inter-block gap.
Claude Code's watchdog measures time since the last completed content block, so it fires at exactly
300 s. On the direct path no block completes early, so there is no inter-block gap and the wait falls
under a different (first-byte) timer.

Impact

A Claude Code session whose next action is a single large tool call (e.g. Write of a multi-thousand-line
file) fails deterministically and unrecoverably behind AuthBridge:

• Every attempt dies at exactly 300 s with API Error: The response stopped arriving.
• It does not self-heal. Claude Code has a retry path for this case, but it is gated on nothing having
  been yielded yet; the early thinking block appears to satisfy that gate and suppress the retry.
• Observed 4/4 failures in one real session over ~45 min. The session became unusable until AuthBridge
  was removed from the path.

Any client with a stream-idle watchdog is affected, not just Claude Code.

Environment

• authbridge-proxy v0.7.0-alpha.7, mode: proxy-sidecar, roles: [forward]
• tls_bridge.mode: enabled, passthrough_hosts: null
• Outbound pipeline: inference-parser, then tool-prune
• Upstream: internal LiteLLM gateway fronting AWS Bedrock (claude-opus-5), reached via
  ANTHROPIC_BASE_URL; forward proxy on 127.0.0.1:47600
• Client: Claude Code 2.1.261, effort: xhigh (so every turn opens with a thinking block)
• macOS 15 (Darwin 25.5.0)

Reproduction

repro.py below. Requires ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN. It issues the same
streaming request with thinking enabled and a forced tool call whose argument is large enough to take
~60 s to generate, once through AuthBridge and once bypassing it, and reports the largest dead-air gap.

#!/usr/bin/env python3
import json, os, subprocess, time, statistics as st

URL = os.environ["ANTHROPIC_BASE_URL"] + "/v1/messages"
TOK = os.environ["ANTHROPIC_AUTH_TOKEN"]
PROXY = "http://127.0.0.1:47600"

TOOLS = [{"name": "save_document", "description": "Save a doc.",
          "input_schema": {"type": "object",
                           "properties": {"path": {"type": "string"},
                                          "content": {"type": "string"}},
                           "required": ["path", "content"]}}]
BODY = json.dumps({
    "model": "claude-opus-5", "max_tokens": 12000, "stream": True,
    "thinking": {"type": "enabled", "budget_tokens": 2048},   # <-- REQUIRED to see the bug
    "tools": TOOLS,
    "messages": [{"role": "user", "content":
        "First think briefly about structure. Then call save_document once, path '/tmp/k.md', "
        "content = a COMPLETE 1500-word technical document on cache invalidation strategies, "
        "6 sections of ~250 words of real prose. Do not abbreviate. No prose outside the tool call."}]})

def run(proxy):
    env = dict(os.environ)
    for k in ("HTTPS_PROXY","HTTP_PROXY","ALL_PROXY","https_proxy","http_proxy","all_proxy"):
        env.pop(k, None)
    if proxy:
        env["HTTPS_PROXY"] = proxy
    cmd = ["curl","-sSk","-N","--max-time","600","-X","POST",URL,
           "-H",f"Authorization: Bearer {TOK}","-H","content-type: application/json",
           "-H","anthropic-version: 2023-06-01","-d",BODY]
    t0 = time.monotonic(); arr = []; n = 0
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, env=env)
    while True:
        c = p.stdout.read(4096)
        if not c: break
        arr.append(time.monotonic() - t0); n += len(c)
    p.wait()
    gaps = [arr[i]-arr[i-1] for i in range(1, len(arr))]
    return arr[0], (max(gaps) if gaps else 0.0), time.monotonic()-t0, len(arr), n

R = {"AB": [], "D": []}
# ABBA counterbalanced so warming drift does not load onto one arm
for i,(lab,px) in enumerate([("AB",PROXY),("D",None),("D",None),
                             ("AB",PROXY),("AB",PROXY),("D",None)], 1):
    ttfb, gap, tot, chunks, nb = run(px)
    R[lab].append((ttfb, gap, tot, chunks, nb))
    print(f"run{i} {'AUTHBRIDGE' if lab=='AB' else 'DIRECT':11} "
          f"ttfb={ttfb:6.2f}s max_gap={gap:6.2f}s total={tot:6.2f}s chunks={chunks} bytes={nb}")
for lab,name in (("AB","AUTHBRIDGE"),("D","DIRECT")):
    v = R[lab]
    print(f"{name:11} median ttfb={st.median([x[0] for x in v]):6.2f}s "
          f"median max_gap={st.median([x[1] for x in v]):6.2f}s")

thinking must be enabled to reproduce. With thinking off there is no leading block to flush
early, no inter-block gap appears, and both paths look identical — see "Why this was missed" below.

Evidence

Delivery shape, thinking enabled, n=3 per arm, ABBA counterbalanced

	max dead-air gap (per run)	median gap	median ttfb
via AuthBridge	55.0 s, 0.1 s, 52.2 s	52.17 s	8.09 s
direct	2.7 s, 0.1 s, 0.0 s	0.09 s	56.84 s

AuthBridge flushes the thinking block early (ttfb ~5–8 s) and then goes silent for the whole
tool-call generation. Direct delivers nothing until the end, so no gap exists.

SSE granularity, identical request bodies

	negotiated HTTP	chunks	wire bytes	input_json_delta events
via AuthBridge	HTTP/2	91–100	369–405 KB	2,553–2,779
direct	HTTP/1.1	5–6	18–20 KB	1

Same payload both ways (~18 KB of document). The 20× wire difference is per-event SSE framing
(~2,700 events × ~140 bytes), not compression. AuthBridge serves the client over HTTP/2; the direct
connection to the gateway negotiates HTTP/1.1.

Real-session correlation

A production session showed 4 failures, each with an identical fingerprint:

18:53:56.421  request sent
18:54:12.920  assistant block recorded (thinking, empty)   <- +16.5s
18:59:12.933  API Error: The response stopped arriving.     <- +300.013s after the block

Gaps measured across the four failures: 300.007 s, 300.181 s, 300.013 s, 300.042 s. That constant
is Claude Code's stream-idle watchdog (max(CLAUDE_STREAM_IDLE_TIMEOUT_MS, 300000)), which measures
from the last completed block — i.e. from the early-flushed thinking block.

Note the transcript's recorded stop_reason: "end_turn", output_tokens: 2, thinking: "" are
client-side synthesised values written when the watchdog fires. They are not what the model
returned, and are misleading when triaging from a transcript alone.

Ruled out (measured, so please don't re-litigate)

• Latency/throughput — 1.4 % on generation throughput (4,285 vs 4,226 B/s; 29.7 vs 29.3 deltas/s).
  Request-side cost on a 312 KB body, n=6/arm ABBA: median 8.12 s vs 7.09 s = +1.03 s, P(slower)=0.64,
  U=23/36, not significant, ranges overlap.
• Response body cap — sized downloads through the bridge succeeded byte-exact to 5 MB
  (tunnel=false, path_not_inference). The response body too large path was never hit.
• tool-prune — fixed 15-name denylist (CronCreate…Workflow); zero overlap with the tools the
  session actually used (Bash, Edit, Read, Write, Skill, AskUserQuestion, WebFetch).
  It is a request-phase mutator only.
• SSE framing corruption — event:/data: line counts, thinking_delta, signature_delta,
  message_delta, message_stop all identical on both paths.

Suggested fix

Preferred: emit SSE keep-alive comment lines (: ping\n\n) during upstream silence on bridged
text/event-stream responses. This is the conventional remedy, needs no upstream change, and protects
every watchdog-bearing client — not just Claude Code.

Alternative: do not flush a leading content block until the next one begins, matching the direct
path's near-atomic delivery. Narrower, but converts the failure back into a first-byte wait, which
clients treat differently (and, for Claude Code, keeps its retry path eligible).

Either way it would be worth documenting that a bridged inference response can sit silent for the whole
duration of a single content block's generation.

Verified vs. inferred

Verified by measurement:

• The gap asymmetry (52.17 s vs 0.09 s median, n=3/arm)
• Early thinking-block flush via AuthBridge; atomic delivery direct
• SSE event-count and HTTP-version difference
• The upstream withholds tool_use until complete on both paths (ttfb 98–190 s with thinking off)
• The 300.0 s constant across four real failures
• All four "ruled out" items above

Inferred, not confirmed — worth checking before relying on it:

• That the early yielded block is what suppresses Claude Code's auto-retry. This is read from the
  client bundle's retry gate (if(!Un && qf===null && …), logging
  Stream idle timeout after thinking-only yield — retrying streaming), not observed directly. No
  retry was seen in any proxied trace, and no direct run was observed retrying either.
• The precise cause of the SSE granularity difference. HTTP/2 vs HTTP/1.1 correlates, but I did not
  isolate it from other differences in how AuthBridge re-originates the upstream connection.

Workarounds

1. CLAUDE_STREAM_IDLE_TIMEOUT_MS=1200000 — the floor is 300 s and the value can only be raised.
   Note CLAUDE_BYTE_STREAM_IDLE_TIMEOUT_MS / CLAUDE_ENABLE_BYTE_WATCHDOG do not help: there
   are no bytes on the wire during the gap.
2. Keep any single tool call under ~250 s of generation (~20 k output tokens at the ~83 tok/s measured).
3. Lower effort so turns do not open with a thinking block.
4. abctl claude-code disable (note: this only strips the proxy from ~/.claude/settings.json; the
   daemon keeps running and serving other clients).