Community llama.cpp-dialect GGUFs of DeepSeek V4 Flash cannot load on ds4f-mxfp4: missing deepseek4.* metadata and tensor-name mismatches
- Lingua principale
- C
- Stelle
- 22.3k
- Fork
- 2.1k
- Merge medio
- 1g 3h
- PR unite (30g)
- 4
Descrizione
DeepSeek V4 Flash ships FP4 experts natively, and `ds4f-mxfp4` serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
`bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0` (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.
Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):
1. **Hard-required metadata keys.** The config loader uses
`required_u32(m, "deepseek4.vocab_size")`-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
`deepseek4.vocab_size` plus eight `deepseek4.*` hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of `tokenizer.ggml.tokens`, and the hc/lora/
hash-layer values from keys the dialect does carry.
2. **Tensor-name mismatches.** About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).
Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.
Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with `--ssd-streaming` for several
weeks, passing a 12/12 eval battery.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.