antirez / antirez/ds4

gguf-tools splicer cannot read the MXFP4 GGUF this repo publishes (unsupported GGML tensor type 39)

Abierto
#693 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

The Flash MXFP4 GGUF (`ds4f-mxfp4`) cannot be used as a donor or base with `gguf-tools/mixed/splice_mixed_expert_layers_gguf.py`, because its `GGML_QUANT_SIZES` table has no entry for type 39.

```
error: unsupported GGML tensor type 39; add it to GGML_QUANT_SIZES
```

Everything else in the splicer already handles it correctly — it deliberately does not require base and donor tensor *types* to match, and it writes each tensor's own `ggml_type` through. Only the block-size table is missing an entry.

One-line fix, using the values from ds4's own table at `ds4.c:2052` (`[39] = {"mxfp4", 32, 17}`):

```python
GGML_QUANT_SIZES = {
...
26: (1, 4, "I32"),
+ 39: (32, 17, "MXFP4"),
}
```

Verified locally with `--dry-run`, splicing MXFP4 routed experts for layers 25-42 onto the IQ2XXS/Q2_K hybrid base:

```
selected donor tensors: 54
selected donor types: MXFP4:54
base tensor payload: 90.88 GiB
mixed tensor payload: 107.76 GiB
```

Motivation, in case it is useful: this produces a quant sized for a single 128 GB Mac. The published Flash quants are 80.8 / 90.9 GiB (fit, leaving ~21 GiB of the 112 GiB wired limit unused) then jump to 145.3 / 153.3 GiB (do not fit). Splicing MXFP4 experts into the upper layers fills that gap at 107.76 GiB.

Happy to open a PR if useful.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.