antirez / antirez/ds4

ds4-server reasoning_content deltas fragmented mid-token at arbitrary byte offsets (OpenAI-compatible streaming)

Đang mở
#685 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

## Summary
Streaming `/v1/chat/completions` emits `reasoning_content` deltas split at arbitrary byte offsets — mid-token, character-level — instead of whole tokens. Reassembled reasoning text is correct and the final `content` answer streams intact, so it's cosmetic for reassembly. But any client that consumes thinking deltas individually (reasoning-length estimation, per-token UIs, signature/checksum checks) gets garbage units, and every delta carries SSE+JSON overhead (wasted tokens).

## Environment
- ds4-server from antirez/ds4, DeepSeek V4 Flash GGUF (`ds4flash.gguf`)
- Launch: `ds4-server --ctx 262144 --port 8181 --model ./ds4flash.gguf`
- Client: live curl to `127.0.0.1:8181/v1/chat/completions`, `stream: true`

## Reproduction / observed
Live request to the local endpoint shows the reasoning stream fragmented at character level:
```
"1. T" → "he use" → "r a" → "sk" → "ed " → "to \"sa" → "y he" → ...
```
Every delta carries empty `"signature":""`. Reassembled text is correct ("1. The user asked to say hello..."); final answer `content` streams intact.

## Expected
Reasoning deltas aligned to token boundaries (one delta per decoded token), mirroring the `content` path.

## Root cause (in ds4-server, not Goose)
`openai_sse_stream_update()` in `ds4_server.c`, `OPENAI_STREAM_THINKING` branch (~line 6385):
```c
const size_t hold = strlen("") - 1;
limit = raw_len > hold ? raw_len - hold : st->emit_pos;
limit = utf8_stream_safe_len(raw, st->emit_pos, limit, false); // UTF-8 safe only
sse_chat_delta_n(fd, r, id, "reasoning_content", raw + st->emit_pos, limit - st->emit_pos);
```
Delta boundaries = accumulated text minus a 7-byte hold-back, clamped only to UTF-8 codepoint boundaries — never token-aligned. Identical shape in the Responses path (~7094) and Anthropic path (~7971). Regression test `test_openai_chat_stream_splits_reasoning_without_tools` (~13973) locks in current behavior.

## Impact
- Clients parsing thinking deltas individually (agent harnesses, reasoning-length counters, stream UIs) see mid-token garbage.
- Per-delta JSON overhead inflates output; "wastes tokens" vs whole-token deltas.
- Harmless for plain chat reassembly.

## Related issues (routing family — distinct from this granularity bug)
- #13, #509, #524, #665 (thinking→reasoning_content routing; #509 explicitly states streaming is "correct" re: labeling)
- #678 (close-tag handling in streaming TEXT mode)
- #681 (don't stop generation inside a thinking block)
- #167, #332, #612, #318 (unclosed/leaked thinking boundaries)

## Suggested fix
Accumulate reasoning per decode step and flush one delta per token (as `content` does), or hold back until the next token boundary; keep `utf8_stream_safe_len` as a floor, not the boundary rule.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.