antirez / antirez/ds4

GLM-5.2 Metal + --ssd-streaming: compact indexer warmup fails for prompts > 8192 tokens ("model range not covered by mapped model views")

Abierto
#891 1 comentario 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

## Summary

On Metal with `--ssd-streaming`, GLM-5.2 (`GLM-5.2-UD-Q2_K_RoutedQ2K.gguf`) fails at prompt-processing time whenever the prompt exceeds the full-attention prefill/work cap (8192 tokens). Prefill reaches 100%, then the compact indexed path fails its warmup:

```
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: Metal model range 2.29..2.29 GiB is not covered by mapped model views
ds4: prompt processing failed: metal GLM compact indexer warmup failed
```

Prompts under the cap work fine on the same setup (generation happily runs past position 8192, so compact indexed *decode* seems fine — only the compact indexed *prefill* warmup fails).

## Environment

- MacBook Pro M5 Max, 128 GB unified memory, internal SSD
- macOS (Darwin 25.5.0), binaries built arm64 native (`arch -arm64 make`)
- ds4 @ c1d4597a80e300b803dc642519718f2c999589da (latest main as of filing)
- Model: `GLM-5.2-UD-Q2_K_RoutedQ2K.gguf` (244 GB, from `./download_model.sh glm-antirez-q2`)

## Repro

```
./ds4 -m gguf/GLM-5.2-UD-Q2_K_RoutedQ2K.gguf --metal --ssd-streaming \
--prompt-file prompt.txt -n 9000 --temp 0.7 -c 32000
```

Results by prompt size (same flags otherwise, only `-c` varied to fit):

| prompt tokens | ctx (`-c`) | result |
|---|---|---|
| 1 827 | 16 000 | ✅ works — 47.5 t/s prefill, 3.1 t/s gen, ran well past position 8192 |
| 5 137 | 18 000 | ✅ works |
| 8 777 | 20 000 | ❌ `metal GLM compact indexer warmup failed` |
| 18 509 | 32 000 | ❌ same |
| 84 449 | 115 000 | ❌ same (prefill reports 100% first) |

The startup log on the working runs shows the boundary explicitly:

```
ds4: GLM session ctx=16000 (model max=1048576); full-attention prefill/work cap=8192; compact indexed decode is used beyond the cap
ds4: GLM compact indexed prefill chunk=4096 score_rows=4096 score_scratch=250.00 MiB
```

## Notes

- DeepSeek V4 Flash on the same machine/build is unaffected: an 84 449-token prompt prefills and generates fine (both the q2 and q2-q4 GGUFs, fully resident in RAM).
- Possibly related: #890 (GLM-5.3-Flash Metal, prompts > 4096, compact indexed path), #595 (CUDA) and #608 (ROCm) GLM prefill failures with `--ssd-streaming`. The "model range X..X GiB is not covered by mapped model views" message (zero-length range) suggests the compact indexer requests a mapping the streaming views never cover.

Happy to run patched builds or add instrumentation if useful — the machine and model stay available.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.