antirez / antirez/ds4

Disk KV cache can be reused across different model weights with the same model_id

未关闭
#805 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.3k
派生
2.1k
平均合并
1 天 3 小时
30 天内合并 PR
4

描述

I found a potentially unsafe disk KV cache reuse case when switching between different GGUF weights that share the same DS4 model structure.

### Setup

- macOS / Metal
- DeepSeek V4 Flash 0731 official GGUF
- Huihui DeepSeek V4 Flash 0731 variant
- Both launched with the same `--kv-disk-dir`
- Same context / prompt format / quantization-compatible cache settings

### Observed behavior

After running the official model and creating an 81,920-token disk checkpoint, I stopped the server, switched `ds4flash.gguf` to the Huihui model, and restarted DS4 using the same KV cache directory.

The Huihui model immediately accepted the checkpoint created by the official model:

```text
kv cache hit text tokens=81920 ... quant=2 ...
```

So a checkpoint produced using one set of model weights can apparently be restored while another set of weights is loaded, as long as the model structure / model_id and other cache compatibility checks match.

### Concern

KV states depend on the actual model weights. Two GGUFs can have the same architecture, tokenizer and DS4 `model_id` while containing different weights, such as fine-tuned or abliterated variants.

In that case the cache can be structurally compatible while semantically invalid, potentially causing silent output corruption rather than an explicit error.

Cross-quant reuse for different quantizations of the same underlying model makes sense, but it seems useful to distinguish that from reuse across genuinely different model weights.

### Possible fix

Could disk KV metadata include a stable model-weight fingerprint, or another model identity derived from the GGUF, and require it to match on restore?

Ideally the identity would remain compatible across supported quantizations of the same underlying weights if cross-quant reuse is intentional.

### Workaround

For now I am using separate cache directories:

```text
/tmp/ds4-kv-official
/tmp/ds4-kv-huihui
```

I can provide additional trace output or test a proposed fix if useful.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。