GLM-5.2 (glm-dsa) prefill fails on ROCm backend initialized on AMD Radeon Graphics (sm_115): "rocm GLM prefill failed" with --ssd-streaming
- 主要语言
- C
- 星标
- 22.4k
- 派生
- 2.1k
- 平均合并
- 1 天 3 小时
- 30 天内合并 PR
- 4
描述
run cli: ./ds4 -m glm52_ds4/models--antirez--GLM-5.2-UD-IQ2_XXS_RoutedIQ2XXS_blk78Q2K.gguf --glm-mtp-timing --temp 0 --ssd-streaming
use make strix-halo build, the free report:
total used free shared buff/cache available
Mem: 31956712 4282560 16017176 77616 12208512 27674152
Swap: 33548404 29060 33519344
gpu mem:15.936G/96.00G
Environment:
Hardware: AMD Radeon Graphics (sm_115), 128 GiB unified memory
Build: make strix-halo, Ubuntu 26.04, rocm is already the newest version (7.2.4.70204-93~24.04).
Model: antirez/GLM-5.2-GGUF → GLM-5.2-UD-IQ2_XXS_RoutedIQ2XXS_blk78Q2K.gguf
ds4: Linux rocm backend set oom_score_adj=1000
ds4: ROCm backend initialized on AMD Radeon Graphics (sm_115)
ds4: SSD streaming auto cache budget
ds4: rocm recommends 96.00 GiB working set
ds4: using 80% total for model + cached experts: 76.80 GiB
ds4: non-routed weights: 22.55 GiB
ds4: routed expert size: 9.28 MiB
ds4: expert budget before prefill/full-layer reserve: 3469 (31.44 GiB)
ds4: GLM ROCm cache capped to 31.44 GiB by the memory guard for ctx=32768
ds4: GLM SSD streaming full resident layers: 0 (auto selected 0 layers)
ds4: rocm SSD streaming total expert budget 31.44 GiB = 5.91 GiB prefill headroom + 25.53 GiB dynamic cache (2817 experts, 9.28 MiB each)
ds4: SSD streaming mixed-precision model: 1/76 routed layers off the slab size class will bypass the expert cache and read experts via mapped model views
ds4: SSD streaming initial rocm model map restricted to token embedding (1 spans, 0.94 GiB tensor span)
ds4: ROCm preparing model tensor mappings: 0.00 GiB
ds4: ROCm startup model preparation covered 0.94 GiB of tensor spans in 0.000s
ds4: rocm backend initialized for graph diagnostics
ds4: memory: KV 5.81 GiB (raw 0.00 + compressed 5.81) + buffers 4.19 GiB + resident model 0.94 GiB + expert cache 25.53 GiB + prefill expert reserve 5.91 GiB = 42.39 GiB planned
ds4: memory detail: ctx=32768 prefill_cap=4096 raw_kv_rows=0 compressed_kv_rows=32768 backend=rocm
ds4: memory: KV 5.81 GiB (raw 0.00 + compressed 5.81) + buffers 4.19 GiB = 10.01 GiB context
ds4: memory detail: ctx=32768 prefill_cap=4096 raw_kv_rows=0 compressed_kv_rows=32768 backend=rocm
ds4: GLM Metal session ctx=32768 (model max=1048576); full-attention prefill/work cap=8192; compact indexed decode is used beyond the cap
ds4: GLM graph using compact DSA KV only; expanded full-attention KV cache is skipped
ds4: GLM graph allocating compact DSA cache: rows=32768 logical_ctx=32768 kv_layers=78 indexer_layers=21 f32 5.81 GiB
ds4: GLM compact indexed prefill chunk=4096 score_rows=2048 score_scratch=256.00 MiB
Commands:
/help Show this help.
/think Use normal thinking mode.
/think-max Use Think Max only when context is at least 393216 tokens.
/nothink Disable thinking mode.
/ctx N Set context size for following prompts.
/power N Set GPU duty cycle percentage, 1..100.
/read FILE Read a prompt from FILE and run it.
/quit, /exit Leave the prompt.
Ctrl+C Stop generation and return to the prompt.
ds4> write a python socket example
processing 23 input tokens: 0/23 (0.0%)ds4: prompt processing failed: rocm GLM prefill failed
ds4>
贡献指南
调研方向
Start by reproducing the exact ./ds4 command from the issue with the strix-halo build and the supplied GLM-5.2 GGUF model. Inspect the ROCm GLM prefill path around the reported "rocm GLM prefill failed" result and compare behavior with and without --ssd-streaming. Done means prompt processing succeeds on the stated AMD Radeon Graphics environment while SSD streaming remains enabled.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, ubuntu
- 领域
- backend, machine-learning
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 冷清
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100