Disk KV cache can be reused across different model weights with the same model_id
- Vorherrschende Sprache
- C
- Sterne
- 22.3k
- Forks
- 2.1k
- Ø Merge
- 1 T. 3 Std.
- Gemergte PRs (30 T.)
- 4
Beschreibung
I found a potentially unsafe disk KV cache reuse case when switching between different GGUF weights that share the same DS4 model structure.
### Setup
- macOS / Metal
- DeepSeek V4 Flash 0731 official GGUF
- Huihui DeepSeek V4 Flash 0731 variant
- Both launched with the same `--kv-disk-dir`
- Same context / prompt format / quantization-compatible cache settings
### Observed behavior
After running the official model and creating an 81,920-token disk checkpoint, I stopped the server, switched `ds4flash.gguf` to the Huihui model, and restarted DS4 using the same KV cache directory.
The Huihui model immediately accepted the checkpoint created by the official model:
```text
kv cache hit text tokens=81920 ... quant=2 ...
```
So a checkpoint produced using one set of model weights can apparently be restored while another set of weights is loaded, as long as the model structure / model_id and other cache compatibility checks match.
### Concern
KV states depend on the actual model weights. Two GGUFs can have the same architecture, tokenizer and DS4 `model_id` while containing different weights, such as fine-tuned or abliterated variants.
In that case the cache can be structurally compatible while semantically invalid, potentially causing silent output corruption rather than an explicit error.
Cross-quant reuse for different quantizations of the same underlying model makes sense, but it seems useful to distinguish that from reuse across genuinely different model weights.
### Possible fix
Could disk KV metadata include a stable model-weight fingerprint, or another model identity derived from the GGUF, and require it to match on restore?
Ideally the identity would remain compatible across supported quantizations of the same underlying weights if cross-quant reuse is intentional.
### Workaround
For now I am using separate cache directories:
```text
/tmp/ds4-kv-official
/tmp/ds4-kv-huihui
```
I can provide additional trace output or test a proposed fix if useful.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.