antirez / antirez/ds4

Community llama.cpp-dialect GGUFs of DeepSeek V4 Flash cannot load on ds4f-mxfp4: missing deepseek4.* metadata and tensor-name mismatches

Abierto
#661 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

DeepSeek V4 Flash ships FP4 experts natively, and `ds4f-mxfp4` serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
`bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0` (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.

Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):

1. **Hard-required metadata keys.** The config loader uses
`required_u32(m, "deepseek4.vocab_size")`-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
`deepseek4.vocab_size` plus eight `deepseek4.*` hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of `tokenizer.ggml.tokens`, and the hc/lora/
hash-layer values from keys the dialect does carry.

2. **Tensor-name mismatches.** About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).

Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.

Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with `--ssd-streaming` for several
weeks, passing a 12/12 eval battery.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.