antirez / antirez/ds4

GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)

Abierto
#620 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

# GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)

## Summary

On Metal SSD streaming, `GLM-5.2-UD-Q4_K_RoutedQ4K.gguf` completes prefill of a ~64.7k-token prompt at ~33 t/s and then fails at the prefill→decode transition. The reported range has **start == end**, i.e. a zero-length lookup:

```
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
```

The request returns an empty response. A short prompt on the same model and the same build works correctly.

This is the same message class as #388 (closed 2026-06-16). This build is newer than that fix, so this looks like a remaining case in the GLM routed path rather than a regression of #388.

## Environment

- ds4 commit `efdadd41e20134af4f3381e1ed90e96fe4faef6f` (2026-07-20), clean checkout, `make` default Metal build
- Apple M5 Max, 128 GiB unified memory, macOS 26.5.2 (25F84)
- `iogpu.wired_limit_mb = 0` (system default)
- Model: `antirez/GLM-5.2-GGUF` → `GLM-5.2-UD-Q4_K_RoutedQ4K.gguf`, 434,170,886,208 bytes, size verified byte-exact against the Hub

## Reproduction

Fails:

```sh
./ds4-server -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf \
--ssd-streaming --ssd-streaming-cache-experts 48GB \
--ctx 90112 --host 127.0.0.1 --port 8000
# then POST /v1/chat/completions with a single user message of ~64,700 tokens
# (German-language document set, "think": true, max_tokens 32768)
```

Server log, abridged:

```
ds4: GLM SSD streaming full resident layers: 0 (auto selected 0 layers)
ds4: metal SSD streaming total expert budget 48.00 GiB = 10.12 GiB prefill headroom + ... dynamic cache
...
ds4-server: chat ctx=0..64709:64709 prefill chunk 2048/64709 (3.2%) avg=27.51 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 59392/64709 (91.8%) avg=34.52 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 64709/64709 (100.0%) avg=33.64 t/s 1923.816s
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
```

Works (same model, same build, same binary):

```sh
./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ssd-streaming \
--ctx 4096 --nothink --tokens 20 -p "Antworte mit einem Satz: Was ist eine Arbeitsunfaehigkeit?"
# -> coherent German sentence, prefill 1.14 t/s, generation 1.73 t/s, no error
```

## What was ruled out

Four configurations, identical failure at the identical point (prefill 100% → decode):

| `--ssd-streaming-cache-experts` | `--ctx` | result |
|---|---|---|
| auto (capped at 12.00 GiB, 94 experts) | 131072 | prefill 42.1 t/s, then the error |
| 64GB | 90112 | rejected at startup by the GLM memory guard (required 100.54 GiB, budget 96.00 GiB) — never reached decode |
| 48GB | 90112 | prefill 33.6 t/s, then the error |
| 32GB + `DS4_GLM_STREAMING_DECODE_FULL_LAYER_MAP=1` | 90112 | prefill 33.5 t/s, then the error |

So the failure is independent of the expert cache budget and of context size, and forcing the full-layer decode map does not avoid it.

## Notes that may help

- The failing range is **zero-length** (`2.29..2.29 GiB`). In `ds4_metal.m` the lookup requires `offset >= view_start && end <= view_end`; a zero-length range at an unmapped offset therefore falls through to the error path. Whether the real defect is the zero-length request itself or a missing view is not something I can tell from outside.
- 2.29 GiB is well past the initial streaming map, which the log reports as `restricted to token embedding (1 spans, 0.94 GiB tensor span)`.
- The file is an Unsloth-dynamic ("UD") quant, i.e. per-layer non-uniform expert quantization — the same property #388 identified as the trigger ("Q4_K islands in an IQ2 sea").
- `QA_BEFORE_RELEASES.md` §7 regression-tests this exact message, but only for the mixed-quant **Flash** GGUF. A GLM routed-Q4 case does not appear to be covered.
- Prefill throughput on this machine is much higher than the README's "one Mac, SSD streaming" row suggests (~33–43 t/s measured vs ~3–5 t/s listed for GLM IQ2_XXS). If that row is stale, it may be worth refreshing.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.