api7 / api7/aisix

messages cross-provider streaming rejects upstream chunks missing `model` (chat tolerates it)

Open
#533 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug priority-normal
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Surfaced while writing the content-capture e2e (#532).

Symptom

A streaming /v1/messages request that routes cross-provider (Anthropic-shape request → OpenAI upstream) fails with a terminal error event when an upstream SSE chunk omits the top-level model field:

event: error
data: {"type":"error","error":{"type":"upstream_decode_error","message":"upstream returned an unparseable body: missing field `model` at line 1 column 105"}}

The OpenAI /v1/chat/completions streaming path, given the same chunks (no model), assembles and returns the response fine.

Why it matters (and why it's low impact in practice)

Real OpenAI streaming chunks always include model, so production traffic is unaffected. But it's a robustness inconsistency: the cross-provider /v1/messages chunk decoder requires model on every chunk while the chat decoder treats it as optional. A provider (or a proxy in front of one) that omits model on some chunks would break cross-provider streaming but not chat — surprising and hard to diagnose.

Repro

Point an /v1/messages model at an OpenAI upstream that streams chunks shaped like {"id":"...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"..."}}]} (no model). The chat path accepts it; the messages cross-provider path returns upstream_decode_error: missing field model.

Suggested fix

Make the chunk struct the cross-provider /v1/messages streaming path deserializes treat model as optional (default empty), matching the chat path's tolerance. Confirm both paths use a consistent chunk model.

Note: not a content-capture bug — content capture sits downstream of this; the request fails before capture runs. Filed separately so #532 stays scoped.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the cross-provider /v1/messages streaming chunk decoder and compare it with the /v1/chat/completions decoder. Add regression coverage using SSE chunks that omit the top-level model field, then verify both streaming paths complete without an upstream_decode_error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.