antirez / antirez/ds4

Disk KV cache can be reused across different model weights with the same model_id

オープン
#805 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

I found a potentially unsafe disk KV cache reuse case when switching between different GGUF weights that share the same DS4 model structure.

### Setup

- macOS / Metal
- DeepSeek V4 Flash 0731 official GGUF
- Huihui DeepSeek V4 Flash 0731 variant
- Both launched with the same `--kv-disk-dir`
- Same context / prompt format / quantization-compatible cache settings

### Observed behavior

After running the official model and creating an 81,920-token disk checkpoint, I stopped the server, switched `ds4flash.gguf` to the Huihui model, and restarted DS4 using the same KV cache directory.

The Huihui model immediately accepted the checkpoint created by the official model:

```text
kv cache hit text tokens=81920 ... quant=2 ...
```

So a checkpoint produced using one set of model weights can apparently be restored while another set of weights is loaded, as long as the model structure / model_id and other cache compatibility checks match.

### Concern

KV states depend on the actual model weights. Two GGUFs can have the same architecture, tokenizer and DS4 `model_id` while containing different weights, such as fine-tuned or abliterated variants.

In that case the cache can be structurally compatible while semantically invalid, potentially causing silent output corruption rather than an explicit error.

Cross-quant reuse for different quantizations of the same underlying model makes sense, but it seems useful to distinguish that from reuse across genuinely different model weights.

### Possible fix

Could disk KV metadata include a stable model-weight fingerprint, or another model identity derived from the GGUF, and require it to match on restore?

Ideally the identity would remain compatible across supported quantizations of the same underlying weights if cross-quant reuse is intentional.

### Workaround

For now I am using separate cache directories:

```text
/tmp/ds4-kv-official
/tmp/ds4-kv-huihui
```

I can provide additional trace output or test a proposed fix if useful.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。