Extend live-prefix rewind (#668) beyond GLM: DeepSeek V4 Flash on Metal still cold-prefills exact-prefix retries
- 主要语言
- C
- 星标
- 22.3k
- 派生
- 2.1k
- 平均合并
- 1 天 3 小时
- 30 天内合并 PR
- 4
描述
## Summary
#668 added live-KV rewind for repeated exact-prefix prompts (commit 7694112), but `live_prefix_rewind_target` is gated on `ds4_engine_is_glm_dsa()`, so DeepSeek V4 Flash on the Metal backend still does a full cold prefill when a client resends a prompt that is an exact token-level prefix of the live session.
## Reproduction / evidence
Agent client (Hermes Agent, OpenAI chat/completions) aborts a stream mid-generation (user redirect). The slot's live KV then holds `prompt + partial generation`. The resent prompt is an exact prefix of that state — and is treated as a total miss:
```
ds4-server: live kv cache miss live=175286 prompt=175249 common=175249 reason=token-mismatch
ds4-server: chat ctx=0..175249:175249 prompt start
...
ds4-server: chat ctx=0..175249:175249 prompt done 643.560s
```
`common == prompt_len` (the whole 175,249-token prompt matches the live prefix; only the 37 partially-generated tokens diverge), yet the slot is discarded and ~10.7 minutes of prefill are redone. This happened 4 times in 15 minutes in a real session.
## Why this seems generalizable
`ds4_session_rewind()` itself looks engine-agnostic (truncates the checkpoint, invalidates MTP/DSpark capture). The GLM gate appears to guard the dense-cache capping (`ds4_session_glm_cap_dense_cache`). Question: what needs validation to enable the rewind for the Flash/Metal session path — is there a Metal-graph or streamed-expert invariant that a truncated checkpoint violates?
## Environment
- macOS, Apple Silicon
- `ds4-server -m ds4flash.gguf -c 262144 --host 127.0.0.1 --port 8001 --kv-disk-dir /tmp/ds4-kv --kv-disk-space-mb 8192 --batched-session 4 --mtp .../DeepSeek-V4-Flash-DSpark-support.gguf --dspark`
- Model: DeepSeek V4 Flash
- Client: Hermes Agent (also reproduced the pattern described in #609)
Related: #609 / PR #611 (OpenAI tool-round continuation), #444 (disk KV eviction). Happy to test any branch — the repro environment is live here.
贡献指南
评估
这个 Issue 还没有评估数据。