antirez / antirez/ds4

ds4-server reasoning_content deltas fragmented mid-token at arbitrary byte offsets (OpenAI-compatible streaming)

Abierto
#685 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

## Summary
Streaming `/v1/chat/completions` emits `reasoning_content` deltas split at arbitrary byte offsets — mid-token, character-level — instead of whole tokens. Reassembled reasoning text is correct and the final `content` answer streams intact, so it's cosmetic for reassembly. But any client that consumes thinking deltas individually (reasoning-length estimation, per-token UIs, signature/checksum checks) gets garbage units, and every delta carries SSE+JSON overhead (wasted tokens).

## Environment
- ds4-server from antirez/ds4, DeepSeek V4 Flash GGUF (`ds4flash.gguf`)
- Launch: `ds4-server --ctx 262144 --port 8181 --model ./ds4flash.gguf`
- Client: live curl to `127.0.0.1:8181/v1/chat/completions`, `stream: true`

## Reproduction / observed
Live request to the local endpoint shows the reasoning stream fragmented at character level:
```
"1. T" → "he use" → "r a" → "sk" → "ed " → "to \"sa" → "y he" → ...
```
Every delta carries empty `"signature":""`. Reassembled text is correct ("1. The user asked to say hello..."); final answer `content` streams intact.

## Expected
Reasoning deltas aligned to token boundaries (one delta per decoded token), mirroring the `content` path.

## Root cause (in ds4-server, not Goose)
`openai_sse_stream_update()` in `ds4_server.c`, `OPENAI_STREAM_THINKING` branch (~line 6385):
```c
const size_t hold = strlen("") - 1;
limit = raw_len > hold ? raw_len - hold : st->emit_pos;
limit = utf8_stream_safe_len(raw, st->emit_pos, limit, false); // UTF-8 safe only
sse_chat_delta_n(fd, r, id, "reasoning_content", raw + st->emit_pos, limit - st->emit_pos);
```
Delta boundaries = accumulated text minus a 7-byte hold-back, clamped only to UTF-8 codepoint boundaries — never token-aligned. Identical shape in the Responses path (~7094) and Anthropic path (~7971). Regression test `test_openai_chat_stream_splits_reasoning_without_tools` (~13973) locks in current behavior.

## Impact
- Clients parsing thinking deltas individually (agent harnesses, reasoning-length counters, stream UIs) see mid-token garbage.
- Per-delta JSON overhead inflates output; "wastes tokens" vs whole-token deltas.
- Harmless for plain chat reassembly.

## Related issues (routing family — distinct from this granularity bug)
- #13, #509, #524, #665 (thinking→reasoning_content routing; #509 explicitly states streaming is "correct" re: labeling)
- #678 (close-tag handling in streaming TEXT mode)
- #681 (don't stop generation inside a thinking block)
- #167, #332, #612, #318 (unclosed/leaked thinking boundaries)

## Suggested fix
Accumulate reasoning per decode step and flush one delta per token (as `content` does), or hold back until the next token boundary; keep `utf8_stream_safe_len` as a floor, not the boundary rule.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.