api7 / api7/aisix

streaming: aborted cross-provider responses and ensemble streams report status 200 instead of 499

Open
#1,004 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug P2
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Two streaming paths report status_code: 200 on usage events for streams the client aborted mid-flight, where the equivalent single-model chat path correctly reports 499 (CLIENT_CLOSED_REQUEST). Both predate the trace-foundation work (verified present at 4a98a83); surfaced during an independent audit of the OTLP trace PR.

Path 1 — cross-provider /v1/responses streaming: the stream guard tracks reached_end, but the status choice only consults guardrail_blocked (422 vs 200), ignoring reached_end == false. An aborted stream therefore emits a terminal usage event with status_code: 200.

Path 2 — streaming ensemble: the judge-stream completion path hardcodes 200 regardless of whether the synthesized answer's stream reached EOF.

Impact: per-status telemetry (usage rows, http.response.status_code on exported spans, status-labelled Prometheus counters) counts aborted streams as successes on these two paths, inconsistent with the single-model chat path.

Expected precedence, matching the chat stream guard: !reached_end → 499; guardrail block → 422; otherwise 200. Worth a paced-stream abort e2e per path when fixing — the existing suites only drain streams to EOF.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the cross-provider /v1/responses streaming guard and the streaming ensemble judge completion path, then compare both with the single-model chat stream guard. Reproduce paced client aborts and verify usage events, exported spans, and Prometheus counters report 499 for incomplete streams, 422 for guardrail blocks, and 200 only after EOF; add an abort e2e for each path if the existing suites support it.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
63/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.