antirez / antirez/ds4

Community llama.cpp-dialect GGUFs of DeepSeek V4 Flash cannot load on ds4f-mxfp4: missing deepseek4.* metadata and tensor-name mismatches

オープン
#661 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

DeepSeek V4 Flash ships FP4 experts natively, and `ds4f-mxfp4` serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
`bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0` (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.

Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):

1. **Hard-required metadata keys.** The config loader uses
`required_u32(m, "deepseek4.vocab_size")`-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
`deepseek4.vocab_size` plus eight `deepseek4.*` hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of `tokenizer.ggml.tokens`, and the hc/lora/
hash-layer values from keys the dialect does carry.

2. **Tensor-name mismatches.** About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).

Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.

Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with `--ssd-streaming` for several
weeks, passing a 12/12 eval battery.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。