ObolNetwork / ObolNetwork/obol-stack
Buyer is charged when an agent upstream fails but returns HTTP 200
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 11
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Summary
A buyer is charged when an agent upstream fails internally but still returns HTTP 200. Observed live on Base mainnet: an agent's LLM provider returned a non-retryable 404, the agent aborted its conversation loop, returned an empty reply with 200 application/json, and the payment settled on-chain. The buyer paid and received nothing.
Root cause
The settle decision is not "status < 400 after the response is known" — on the non-SSE path it happens before the body exists at all.
settlementInterceptor.WriteHeadercallssettleFunc()atinternal/x402/forwardauth.go:1311, i.e. the momentReverseProxyhands over the upstream response headers. The only gate before it isstatusCode >= 400(internal/x402/forwardauth.go:1286).- The body-liveness guard that would catch an empty answer,
bytesWritten == 0(internal/x402/forwardauth.go:1347), is unreachable on this path:finalizereturns immediately unlessstreamDeferis set (internal/x402/forwardauth.go:1324), andstreamDeferis only set fortext/event-stream(internal/x402/forwardauth.go:1297-1305).
So any upstream that reports its own failure inside a 200 is settled as a success.
Both request shapes are affected
stream: false— settles atWriteHeader, before any body byte. This is the shape that produced the incident.stream: true(SSE) — reachesfinalize, but a role-only delta plusdata: [DONE]is roughly 60 bytes, which clearsbytesWritten == 0and settles too.
A fix that covers only one shape is not a fix.
It is recorded as a clean sale
chargedRequests increments (internal/x402/verifier.go:495-498) while upstreamFailedAfterVerify does not (internal/x402/verifier.go:499-503 requires status >= 400), so the failure is invisible in metrics.
Related defect found while investigating
settlementInterceptor.Flush (internal/x402/forwardauth.go:1362-1366) is an unconditional passthrough, and statusRecorder.Flush (internal/x402/verifier.go:1020-1024) forwards it to the real writer. httputil sets flushInterval = -1 for any response with ContentLength == -1 — chunked, not only SSE — and copyResponse arms delayedFlush before the first body byte.
A chunked application/json reply therefore commits an implicit 200 while wroteStatus is still false, which:
- silently drops the
X-PAYMENT-RESPONSE/PAYMENT-RESPONSEreceipt set later atinternal/x402/forwardauth.go:862-865, leaving the buyer to log "auth consumed without observed settlement" (internal/x402/buyer/proxy.go:741-748) — charged on-chain with no receipt to reconcile against; - defeats the
wroteStatusguards atinternal/x402/forwardauth.go:826and:847, so a settle failure cannot flip the status to 503.
No existing test catches this: httptest handlers with small bodies get an automatic Content-Length, so flushInterval is 0. Any buffering scheme added here must gate Flush.
Proposed direction
Settle only when the upstream actually delivered a completion, behind an opt-in ForwardAuthConfig flag so existing behaviour is unchanged by default:
- For non-SSE chat-shaped routes, hold status and body until
finalizeand decide there. - For SSE, tap the stream to detect whether any content delta was delivered — without buffering or delaying a byte, so incremental streaming timing is preserved.
- Treat a zero-byte body as "no value" before attempting to parse it —
json.Unmarshal([]byte(""), …)returns an error, so a naive parse-then-fail-open settles the exact empty-body case in this report. - Gate
Flushunder any buffering mode (see above).
Two-phase auth-capture (authorize now, capture once the answer is known good) is not available as an alternative: the facilitator rejects it with ERR_TWO_PHASE_NOT_SUPPORTED, and autoCapture: true is hardcoded at internal/x402/authcapture.go:142.
Impact
Any paid agent or inference offer whose upstream can fail while returning 2xx. With the platform fee enabled, the fee splits on-chain for the failed request too.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with settlementInterceptor.WriteHeader, finalize, and Flush in internal/x402/forwardauth.go, then trace statusRecorder.Flush and the chargedRequests metrics in internal/x402/verifier.go. Reproduce both non-SSE and SSE paths, including chunked application/json responses, and add coverage showing settlement and receipt handling occur only after a valid completion while preserving streaming behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend-api-design, payments
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100