antirez / antirez/ds4

Community llama.cpp-dialect GGUFs of DeepSeek V4 Flash cannot load on ds4f-mxfp4: missing deepseek4.* metadata and tensor-name mismatches

未关闭
#661 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.3k
派生
2.1k
平均合并
1 天 3 小时
30 天内合并 PR
4

描述

DeepSeek V4 Flash ships FP4 experts natively, and `ds4f-mxfp4` serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
`bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0` (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.

Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):

1. **Hard-required metadata keys.** The config loader uses
`required_u32(m, "deepseek4.vocab_size")`-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
`deepseek4.vocab_size` plus eight `deepseek4.*` hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of `tokenizer.ggml.tokens`, and the hc/lora/
hash-layer values from keys the dialect does carry.

2. **Tensor-name mismatches.** About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).

Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.

Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with `--ssd-streaming` for several
weeks, passing a 12/12 eval battery.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。