GLM-5.2 Metal + --ssd-streaming: compact indexer warmup fails for prompts > 8192 tokens ("model range not covered by mapped model views")
- 主要言語
- C
- スター
- 22.3k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
## Summary
On Metal with `--ssd-streaming`, GLM-5.2 (`GLM-5.2-UD-Q2_K_RoutedQ2K.gguf`) fails at prompt-processing time whenever the prompt exceeds the full-attention prefill/work cap (8192 tokens). Prefill reaches 100%, then the compact indexed path fails its warmup:
```
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: prompt processing failed: metal GLM compact indexer warmup failed
```
Prompts under the cap work fine on the same setup (generation happily runs past position 8192, so compact indexed *decode* seems fine — only the compact indexed *prefill* warmup fails).
## Environment
- MacBook Pro M5 Max, 128 GB unified memory, internal SSD
- macOS (Darwin 25.5.0), binaries built arm64 native (`arch -arm64 make`)
- ds4 @ c1d4597a80e300b803dc642519718f2c999589da (latest main as of filing)
- Model: `GLM-5.2-UD-Q2_K_RoutedQ2K.gguf` (244 GB, from `./download_model.sh glm-antirez-q2`)
## Repro
```
./ds4 -m gguf/GLM-5.2-UD-Q2_K_RoutedQ2K.gguf --metal --ssd-streaming \
--prompt-file prompt.txt -n 9000 --temp 0.7 -c 32000
```
Results by prompt size (same flags otherwise, only `-c` varied to fit):
| prompt tokens | ctx (`-c`) | result |
|---|---|---|
| 1 827 | 16 000 | ✅ works — 47.5 t/s prefill, 3.1 t/s gen, ran well past position 8192 |
| 5 137 | 18 000 | ✅ works |
| 8 777 | 20 000 | ❌ `metal GLM compact indexer warmup failed` |
| 18 509 | 32 000 | ❌ same |
| 84 449 | 115 000 | ❌ same (prefill reports 100% first) |
The startup log on the working runs shows the boundary explicitly:
```
ds4: GLM session ctx=16000 (model max=1048576); full-attention prefill/work cap=8192; compact indexed decode is used beyond the cap
ds4: GLM compact indexed prefill chunk=4096 score_rows=4096 score_scratch=250.00 MiB
```
## Notes
- DeepSeek V4 Flash on the same machine/build is unaffected: an 84 449-token prompt prefills and generates fine (both the q2 and q2-q4 GGUFs, fully resident in RAM).
- Possibly related: #890 (GLM-5.3-Flash Metal, prompts > 4096, compact indexed path), #595 (CUDA) and #608 (ROCm) GLM prefill failures with `--ssd-streaming`. The "model range X..X GiB is not covered by mapped model views" message (zero-length range) suggests the compact indexer requests a mapping the streaming views never cover.
Happy to run patched builds or add instrumentation if useful — the machine and model stay available.
コントリビューションガイド
評価
この issue はまだ評価されていません。