GLM-5.2 Q4: prefill fails at ≥64K context on M3 Ultra 512GB (KV payload staging failed)
- Dominant language
- C
- Stars
- 22.3k
- Forks
- 2.1k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 4
Description
## Environment
- **Hardware**: Apple M3 Ultra, 512GB unified memory, 819GB/s bandwidth
- **Model**: GLM-5.2-UD-Q4_K_RoutedQ4K.gguf (404GB, from antirez/GLM-5.2-GGUF on HuggingFace)
- **Engine**: ds4-server, built from latest main (Metal backend)
- **Context**: --ctx 524288 (512K)
- **Command**: ./ds4-server -m GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ctx 524288 --port 8321 --kv-disk-dir ~/.ds4/server-kv --kv-disk-space-mb 262144
## What works
- Short context (2K, 16K): prefill and decode work correctly
- 2K: prefill 116 t/s, decode 16.0 t/s
- 16K: prefill 112 t/s, decode 14.2 t/s
- Memory is sufficient (model 404GB + graph ~50GB = ~455GB, well within 512GB)
- Quality: needle tests pass when thinking mode is disabled
## What fails
At ≥64K context, prefill fails with:
```
ds4-server: kv cache skipped tokens=10240 reason=continued because KV payload staging failed: se...
ds4-server: kv cache discarded reason=prefill-failed
```
After the failure, the engine stops responding to new requests (appears hung).
Reproducible on a fresh process. Same hardware runs DeepSeek V4 Flash MXFP4 at 1M context without issues.
## Log excerpt
```
0907 02:11:17 ds4-server: kv cache skipped tokens=10240 reason=continued because KV payload staging failed: se...
0907 02:35:35 ds4-server: chat ctx=2048..79031:76983 prompt start
0907 02:35:35 ds4-server: kv cache discarded reason=prefill-failed file=...kv
```
## Notes
- This might be related to GLM's IndexShare architecture or the compact DSA KV layout at larger contexts
- The model loads fine and memory guard passes at 512K ctx (required: ~455GB, budget: 480GB)
- Happy to provide more logs or test patches
Contributor guide
Assessment
This issue has not been assessed yet.