antirez / antirez/ds4

GLM-5.2 Metal + --ssd-streaming: compact indexer warmup fails for prompts > 8192 tokens ("model range not covered by mapped model views")

オープン
#891 コメント 1 件 リアクション 1 件 担当者 0 名 GitHub で見る
主要言語
C
スター
22.3k
フォーク
2.1k
平均マージ
1日 3時間
マージ済み PR(30日)
4

説明

## Summary

On Metal with `--ssd-streaming`, GLM-5.2 (`GLM-5.2-UD-Q2_K_RoutedQ2K.gguf`) fails at prompt-processing time whenever the prompt exceeds the full-attention prefill/work cap (8192 tokens). Prefill reaches 100%, then the compact indexed path fails its warmup:

```
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: prompt processing failed: metal GLM compact indexer warmup failed
```

Prompts under the cap work fine on the same setup (generation happily runs past position 8192, so compact indexed *decode* seems fine — only the compact indexed *prefill* warmup fails).

## Environment

- MacBook Pro M5 Max, 128 GB unified memory, internal SSD
- macOS (Darwin 25.5.0), binaries built arm64 native (`arch -arm64 make`)
- ds4 @ c1d4597a80e300b803dc642519718f2c999589da (latest main as of filing)
- Model: `GLM-5.2-UD-Q2_K_RoutedQ2K.gguf` (244 GB, from `./download_model.sh glm-antirez-q2`)

## Repro

```
./ds4 -m gguf/GLM-5.2-UD-Q2_K_RoutedQ2K.gguf --metal --ssd-streaming \
--prompt-file prompt.txt -n 9000 --temp 0.7 -c 32000
```

Results by prompt size (same flags otherwise, only `-c` varied to fit):

| prompt tokens | ctx (`-c`) | result |
|---|---|---|
| 1 827 | 16 000 | ✅ works — 47.5 t/s prefill, 3.1 t/s gen, ran well past position 8192 |
| 5 137 | 18 000 | ✅ works |
| 8 777 | 20 000 | ❌ `metal GLM compact indexer warmup failed` |
| 18 509 | 32 000 | ❌ same |
| 84 449 | 115 000 | ❌ same (prefill reports 100% first) |

The startup log on the working runs shows the boundary explicitly:

```
ds4: GLM session ctx=16000 (model max=1048576); full-attention prefill/work cap=8192; compact indexed decode is used beyond the cap
ds4: GLM compact indexed prefill chunk=4096 score_rows=4096 score_scratch=250.00 MiB
```

## Notes

- DeepSeek V4 Flash on the same machine/build is unaffected: an 84 449-token prompt prefills and generates fine (both the q2 and q2-q4 GGUFs, fully resident in RAM).
- Possibly related: #890 (GLM-5.3-Flash Metal, prompts > 4096, compact indexed path), #595 (CUDA) and #608 (ROCm) GLM prefill failures with `--ssd-streaming`. The "model range X..X GiB is not covered by mapped model views" message (zero-length range) suggests the compact indexer requests a mapping the streaming views never cover.

Happy to run patched builds or add instrumentation if useful — the machine and model stay available.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。