anomalyco / anomalyco/opencode

ai: preserve diagnostics for mid-stream HTTP decode failures

Open
#42,482 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Summary

RequestExecutor.httpError discards the native cause of an HTTP response-body read failure and omits safe stream-progress metadata. A mid-stream provider failure is therefore persisted and logged only as a generic Decode error (200 POST ...), making the underlying transport failure irretrievable.

Mentioning @kitlangton, who observed and investigated the incident.

Environment

  • opencode version: 0.0.0-next-17430
  • OS: macOS 25.5.0, Darwin arm64
  • Terminal: Ghostty 1.3.1 (TERM_PROGRAM=ghostty)
  • Shell: /bin/zsh
  • Install/channel: next
  • Active plugins: session-control.ts, opencode-voice, and the V2 debug workspace plugin

Reproduction

The production incident was intermittent, but the loss is deterministic with a test executor:

  1. Return an HTTP 200 response whose body stream emits one or more valid chunks.
  2. Reject the ReadableStream after those chunks with a nested native error or a cause that does not decode as { message: string }.
  3. Consume the response through RequestExecutor.stream.
  4. Inspect the resulting AIError, service log, and persisted session error.

Expected Behavior

The failure should retain safe, structured diagnostics without retaining response payloads:

  • HTTP operation and status
  • Normalized native error tag/class and code
  • Bounded cause-chain tags/codes
  • Chunks and bytes successfully read
  • Decoded event count and last event type
  • Whether a terminal protocol event was observed
  • Content type/encoding and elapsed time

The native payload, generated text, reasoning content, tool arguments, credentials, arbitrary headers, and raw SSE frames should not be logged or persisted.

Actual Behavior

The assistant message was persisted with only:

{
  "type": "provider.transport",
  "message": "Decode error (200 POST https://chatgpt.com/backend-api/codex/responses)"
}

The service log contained the same generic message as both the AIError and nested cause. Nine reasoning-summary events had been persisted before the body read failed, but no answer text or terminal response event arrived. A later retry succeeded and the service remained healthy.

Additional Context

The information is lost along this path on current v2:

  1. Effect captures the native ReadableStream rejection as DecodeError.cause in effect/unstable/http/HttpClientResponse.ts.
  2. packages/ai/src/route/executor.ts unwraps reason.cause, but nativeTransportFailure only decodes a shallow { message, code, cause } shape.
  3. If decoding the source fails, httpError falls back to input.error.message, the outer generic decode message, rather than inspecting a bounded cause chain.
  4. httpError constructs a replacement AIError without the original causal chain and includes only request context for read failures, despite the HttpClientError carrying the response.
  5. packages/core/src/session/to-session-error.ts deliberately persists only the projected session error type, message, and optional status.

Suggested minimal changes:

  1. Recursively inspect a bounded cause chain and retain only normalized tags/classes and codes.
  2. Include allowlisted response metadata and HTTP status for read failures.
  3. Count chunks/bytes at RequestExecutor.stream and decoded event metadata at the protocol boundary without retaining payloads.
  4. Emit one structured sanitized diagnostic when a stream fails, including failures that will be retried.
  5. Add a regression test for a successful HTTP 200 stream that emits valid chunks and then fails.
  6. Replace arbitrary redacted diagnostic headers/query values with explicit allowlists.

Raw response retention is intentionally not proposed: truncation limits size, not sensitivity, and even a short SSE frame may contain generated text, prompts, tool arguments, credentials, account identifiers, or encrypted reasoning.

Related but distinct: #41320 reports a Cloudflare 1010 disconnection. This issue concerns preservation of sanitized diagnostics after any mid-stream response-body failure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading packages/ai/src/route/executor.ts, effect/unstable/http/HttpClientResponse.ts, and packages/core/src/session/to-session-error.ts to trace the failure and persistence paths. Define bounded, allowlisted diagnostics that exclude payloads and sensitive values, then add the described regression test for a 200 stream that emits valid chunks before failing. Done means sanitized structured diagnostics preserve transport and stream-progress context through logging and session errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend, observability, security, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.