chat/completions: streamed logprobs are dropped during reassembly — only last chunk survives
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 33
- Avg merge
- 45m
- Merged PRs (30d)
- 10
Description
Summary
When a client requests logprobs: true with stream: true, the non-streaming result (and the SSE stream) only contains the logprobs of the last output token. All intermediate token logprobs are lost.
Repro
Request (official DeepSeek API as upstream):
POST /v1/chat/completions
{ "model": "deepseek-v4-flash", "messages": [{ "role": "user", "content": "Reply with exactly: alpha beta gamma delta" }], "thinking": {"type":"disabled"}, "logprobs": true, "top_logprobs": 3, "max_tokens": 100, "stream": true }
Direct to upstream: 6 chunks, 4 content tokens, all 4 carry logprobs ✅
Through Floway: 3 chunks, only the final chunk carries logprobs (1-2 tokens) ❌
Same with non-streaming: Floway returns logprobs.content with 1 token instead of 4.
Root cause
packages/protocols/src/chat-completions/reassemble.ts:
KNOWN_DELTA_KEYSdoes not includelogprobs→ it falls through tocaptureExtrascaptureExtrasuses last-write-wins for non-string non-array values → each chunk overwrites the previous logprobs, leaving only the final chunk's
Same pattern exists in packages/protocols/src/completions/reassemble.ts (accumulator.logprobs = choice.logprobs overwrite).
Expected
Logprobs accumulate across chunks (append to the content array), so logprobs.content has one entry per output token, matching the non-streaming response. OpenAI's streaming wire format sends logprobs per-token in each chunk's delta.logprobs.content — they should be concatenated, not replaced.
Environment
Floway via pnpm workspace monorepo (gateway + protocols packages), upstream = DeepSeek official API (api.deepseek.com, chatCompletions endpoint).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/protocols/src/chat-completions/reassemble.ts, tracing KNOWN_DELTA_KEYS and captureExtras, then compare the overwrite in packages/protocols/src/completions/reassemble.ts. Use the supplied streaming and non-streaming DeepSeek reproductions to verify that logprobs.content accumulates one entry per output token rather than retaining only the final chunk.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend-api-design
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100