antirez / antirez/ds4

ds4-server reasoning_content deltas fragmented mid-token at arbitrary byte offsets (OpenAI-compatible streaming)

オープン
#685 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

## Summary
Streaming `/v1/chat/completions` emits `reasoning_content` deltas split at arbitrary byte offsets — mid-token, character-level — instead of whole tokens. Reassembled reasoning text is correct and the final `content` answer streams intact, so it's cosmetic for reassembly. But any client that consumes thinking deltas individually (reasoning-length estimation, per-token UIs, signature/checksum checks) gets garbage units, and every delta carries SSE+JSON overhead (wasted tokens).

## Environment
- ds4-server from antirez/ds4, DeepSeek V4 Flash GGUF (`ds4flash.gguf`)
- Launch: `ds4-server --ctx 262144 --port 8181 --model ./ds4flash.gguf`
- Client: live curl to `127.0.0.1:8181/v1/chat/completions`, `stream: true`

## Reproduction / observed
Live request to the local endpoint shows the reasoning stream fragmented at character level:
```
"1. T" → "he use" → "r a" → "sk" → "ed " → "to \"sa" → "y he" → ...
```
Every delta carries empty `"signature":""`. Reassembled text is correct ("1. The user asked to say hello..."); final answer `content` streams intact.

## Expected
Reasoning deltas aligned to token boundaries (one delta per decoded token), mirroring the `content` path.

## Root cause (in ds4-server, not Goose)
`openai_sse_stream_update()` in `ds4_server.c`, `OPENAI_STREAM_THINKING` branch (~line 6385):
```c
const size_t hold = strlen("") - 1;
limit = raw_len > hold ? raw_len - hold : st->emit_pos;
limit = utf8_stream_safe_len(raw, st->emit_pos, limit, false); // UTF-8 safe only
sse_chat_delta_n(fd, r, id, "reasoning_content", raw + st->emit_pos, limit - st->emit_pos);
```
Delta boundaries = accumulated text minus a 7-byte hold-back, clamped only to UTF-8 codepoint boundaries — never token-aligned. Identical shape in the Responses path (~7094) and Anthropic path (~7971). Regression test `test_openai_chat_stream_splits_reasoning_without_tools` (~13973) locks in current behavior.

## Impact
- Clients parsing thinking deltas individually (agent harnesses, reasoning-length counters, stream UIs) see mid-token garbage.
- Per-delta JSON overhead inflates output; "wastes tokens" vs whole-token deltas.
- Harmless for plain chat reassembly.

## Related issues (routing family — distinct from this granularity bug)
- #13, #509, #524, #665 (thinking→reasoning_content routing; #509 explicitly states streaming is "correct" re: labeling)
- #678 (close-tag handling in streaming TEXT mode)
- #681 (don't stop generation inside a thinking block)
- #167, #332, #612, #318 (unclosed/leaked thinking boundaries)

## Suggested fix
Accumulate reasoning per decode step and flush one delta per token (as `content` does), or hold back until the next token boundary; keep `utf8_stream_safe_len` as a floor, not the boundary rule.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。