GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)
- 主要言語
- C
- スター
- 22.3k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
# GLM-5.2 routed-Q4: decode fails after a long prefill with "Metal model range X..X GiB is not covered by mapped model views" (zero-length range)
## Summary
On Metal SSD streaming, `GLM-5.2-UD-Q4_K_RoutedQ4K.gguf` completes prefill of a ~64.7k-token prompt at ~33 t/s and then fails at the prefill→decode transition. The reported range has **start == end**, i.e. a zero-length lookup:
```
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
```
The request returns an empty response. A short prompt on the same model and the same build works correctly.
This is the same message class as #388 (closed 2026-06-16). This build is newer than that fix, so this looks like a remaining case in the GLM routed path rather than a regression of #388.
## Environment
- ds4 commit `efdadd41e20134af4f3381e1ed90e96fe4faef6f` (2026-07-20), clean checkout, `make` default Metal build
- Apple M5 Max, 128 GiB unified memory, macOS 26.5.2 (25F84)
- `iogpu.wired_limit_mb = 0` (system default)
- Model: `antirez/GLM-5.2-GGUF` → `GLM-5.2-UD-Q4_K_RoutedQ4K.gguf`, 434,170,886,208 bytes, size verified byte-exact against the Hub
## Reproduction
Fails:
```sh
./ds4-server -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf \
--ssd-streaming --ssd-streaming-cache-experts 48GB \
--ctx 90112 --host 127.0.0.1 --port 8000
# then POST /v1/chat/completions with a single user message of ~64,700 tokens
# (German-language document set, "think": true, max_tokens 32768)
```
Server log, abridged:
```
ds4: GLM SSD streaming full resident layers: 0 (auto selected 0 layers)
ds4: metal SSD streaming total expert budget 48.00 GiB = 10.12 GiB prefill headroom + ... dynamic cache
...
ds4-server: chat ctx=0..64709:64709 prefill chunk 2048/64709 (3.2%) avg=27.51 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 59392/64709 (91.8%) avg=34.52 t/s
ds4-server: chat ctx=0..64709:64709 prefill chunk 64709/64709 (100.0%) avg=33.64 t/s 1923.816s
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
```
Works (same model, same build, same binary):
```sh
./ds4 -m gguf/GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ssd-streaming \
--ctx 4096 --nothink --tokens 20 -p "Antworte mit einem Satz: Was ist eine Arbeitsunfaehigkeit?"
# -> coherent German sentence, prefill 1.14 t/s, generation 1.73 t/s, no error
```
## What was ruled out
Four configurations, identical failure at the identical point (prefill 100% → decode):
| `--ssd-streaming-cache-experts` | `--ctx` | result |
|---|---|---|
| auto (capped at 12.00 GiB, 94 experts) | 131072 | prefill 42.1 t/s, then the error |
| 64GB | 90112 | rejected at startup by the GLM memory guard (required 100.54 GiB, budget 96.00 GiB) — never reached decode |
| 48GB | 90112 | prefill 33.6 t/s, then the error |
| 32GB + `DS4_GLM_STREAMING_DECODE_FULL_LAYER_MAP=1` | 90112 | prefill 33.5 t/s, then the error |
So the failure is independent of the expert cache budget and of context size, and forcing the full-layer decode map does not avoid it.
## Notes that may help
- The failing range is **zero-length** (`2.29..2.29 GiB`). In `ds4_metal.m` the lookup requires `offset >= view_start && end <= view_end`; a zero-length range at an unmapped offset therefore falls through to the error path. Whether the real defect is the zero-length request itself or a missing view is not something I can tell from outside.
- 2.29 GiB is well past the initial streaming map, which the log reports as `restricted to token embedding (1 spans, 0.94 GiB tensor span)`.
- The file is an Unsloth-dynamic ("UD") quant, i.e. per-layer non-uniform expert quantization — the same property #388 identified as the trigger ("Q4_K islands in an IQ2 sea").
- `QA_BEFORE_RELEASES.md` §7 regression-tests this exact message, but only for the mixed-quant **Flash** GGUF. A GLM routed-Q4 case does not appear to be covered.
- Prefill throughput on this machine is much higher than the README's "one Mac, SSD streaming" row suggests (~33–43 t/s measured vs ~3–5 t/s listed for GLM IQ2_XXS). If that row is stale, it may be worth refreshing.
コントリビューションガイド
評価
この issue はまだ評価されていません。