antirez / antirez/ds4

Community llama.cpp-dialect GGUFs of DeepSeek V4 Flash cannot load on ds4f-mxfp4: missing deepseek4.* metadata and tensor-name mismatches

Đang mở
#661 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

DeepSeek V4 Flash ships FP4 experts natively, and `ds4f-mxfp4` serves them --
but only from GGUFs produced by its own quantizer from the original
safetensors. Community GGUF exports in the llama.cpp dialect, e.g.
`bullerwins/DeepSeek-V4-Flash-0731-MXFP4_MOE-Q8_0` (156 GB), do not load at
all, even though their MXFP4 routed-expert payload is exactly what this
branch was built to serve.

Two independent gaps on the metadata/naming side (part 1 of 2; the dense
tensor type gap is filed separately):

1. **Hard-required metadata keys.** The config loader uses
`required_u32(m, "deepseek4.vocab_size")`-style hard requirements
(ds4.c ~5613 on this branch). The community dialect omits
`deepseek4.vocab_size` plus eight `deepseek4.*` hc/lora/hash-layer keys,
so the load dies at the first missing key. All of them are derivable:
vocab_size from the length of `tokenizer.ggml.tokens`, and the hc/lora/
hash-layer values from keys the dialect does carry.

2. **Tensor-name mismatches.** About twenty tensor lookups fail because the
community naming differs from ds4 canonical names (missing suffixes,
renamed attention and expert tensors).

Both are addressable without touching the native path: derivation only runs
when a required key is absent, and the alias table is consulted only when
the canonical lookup misses, so GGUFs from the branch's own quantizer load
exactly as before.

Evidence this closes a real gap: with the compat layer applied (plus the
dense type conversion in the companion issue), the bullerwins GGUF has been
serving in production on a GB10 CUDA node with `--ssd-streaming` for several
weeks, passing a 12/12 eval battery.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.