antirez / antirez/ds4

gguf-tools splicer cannot read the MXFP4 GGUF this repo publishes (unsupported GGML tensor type 39)

オープン
#693 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

The Flash MXFP4 GGUF (`ds4f-mxfp4`) cannot be used as a donor or base with `gguf-tools/mixed/splice_mixed_expert_layers_gguf.py`, because its `GGML_QUANT_SIZES` table has no entry for type 39.

```
error: unsupported GGML tensor type 39; add it to GGML_QUANT_SIZES
```

Everything else in the splicer already handles it correctly — it deliberately does not require base and donor tensor *types* to match, and it writes each tensor's own `ggml_type` through. Only the block-size table is missing an entry.

One-line fix, using the values from ds4's own table at `ds4.c:2052` (`[39] = {"mxfp4", 32, 17}`):

```python
GGML_QUANT_SIZES = {
...
26: (1, 4, "I32"),
+ 39: (32, 17, "MXFP4"),
}
```

Verified locally with `--dry-run`, splicing MXFP4 routed experts for layers 25-42 onto the IQ2XXS/Q2_K hybrid base:

```
selected donor tensors: 54
selected donor types: MXFP4:54
base tensor payload: 90.88 GiB
mixed tensor payload: 107.76 GiB
```

Motivation, in case it is useful: this produces a quant sized for a single 128 GB Mac. The published Flash quants are 80.8 / 90.9 GiB (fit, leaving ~21 GiB of the 112 GiB wired limit unused) then jump to 145.3 / 153.3 GiB (do not fit). Splicing MXFP4 experts into the upper layers fills that gap at 107.76 GiB.

Happy to open a PR if useful.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。