antirez / antirez/ds4

GLM-5.2 (glm-dsa) prefill fails on ROCm backend initialized on AMD Radeon Graphics (sm_115): "rocm GLM prefill failed" with --ssd-streaming

Đang mở
#608 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

run cli: ./ds4 -m glm52_ds4/models--antirez--GLM-5.2-UD-IQ2_XXS_RoutedIQ2XXS_blk78Q2K.gguf --glm-mtp-timing --temp 0 --ssd-streaming
use make strix-halo build, the free report:
total used free shared buff/cache available
Mem: 31956712 4282560 16017176 77616 12208512 27674152
Swap: 33548404 29060 33519344

gpu mem:15.936G/96.00G

Environment:
Hardware: AMD Radeon Graphics (sm_115), 128 GiB unified memory
Build: make strix-halo, Ubuntu 26.04, rocm is already the newest version (7.2.4.70204-93~24.04).
Model: antirez/GLM-5.2-GGUF → GLM-5.2-UD-IQ2_XXS_RoutedIQ2XXS_blk78Q2K.gguf

ds4: Linux rocm backend set oom_score_adj=1000
ds4: ROCm backend initialized on AMD Radeon Graphics (sm_115)
ds4: SSD streaming auto cache budget
ds4: rocm recommends 96.00 GiB working set
ds4: using 80% total for model + cached experts: 76.80 GiB
ds4: non-routed weights: 22.55 GiB
ds4: routed expert size: 9.28 MiB
ds4: expert budget before prefill/full-layer reserve: 3469 (31.44 GiB)
ds4: GLM ROCm cache capped to 31.44 GiB by the memory guard for ctx=32768
ds4: GLM SSD streaming full resident layers: 0 (auto selected 0 layers)
ds4: rocm SSD streaming total expert budget 31.44 GiB = 5.91 GiB prefill headroom + 25.53 GiB dynamic cache (2817 experts, 9.28 MiB each)
ds4: SSD streaming mixed-precision model: 1/76 routed layers off the slab size class will bypass the expert cache and read experts via mapped model views
ds4: SSD streaming initial rocm model map restricted to token embedding (1 spans, 0.94 GiB tensor span)
ds4: ROCm preparing model tensor mappings: 0.00 GiB
ds4: ROCm startup model preparation covered 0.94 GiB of tensor spans in 0.000s
ds4: rocm backend initialized for graph diagnostics
ds4: memory: KV 5.81 GiB (raw 0.00 + compressed 5.81) + buffers 4.19 GiB + resident model 0.94 GiB + expert cache 25.53 GiB + prefill expert reserve 5.91 GiB = 42.39 GiB planned
ds4: memory detail: ctx=32768 prefill_cap=4096 raw_kv_rows=0 compressed_kv_rows=32768 backend=rocm
ds4: memory: KV 5.81 GiB (raw 0.00 + compressed 5.81) + buffers 4.19 GiB = 10.01 GiB context
ds4: memory detail: ctx=32768 prefill_cap=4096 raw_kv_rows=0 compressed_kv_rows=32768 backend=rocm
ds4: GLM Metal session ctx=32768 (model max=1048576); full-attention prefill/work cap=8192; compact indexed decode is used beyond the cap
ds4: GLM graph using compact DSA KV only; expanded full-attention KV cache is skipped
ds4: GLM graph allocating compact DSA cache: rows=32768 logical_ctx=32768 kv_layers=78 indexer_layers=21 f32 5.81 GiB
ds4: GLM compact indexed prefill chunk=4096 score_rows=2048 score_scratch=256.00 MiB
Commands:
/help Show this help.
/think Use normal thinking mode.
/think-max Use Think Max only when context is at least 393216 tokens.
/nothink Disable thinking mode.
/ctx N Set context size for following prompts.
/power N Set GPU duty cycle percentage, 1..100.
/read FILE Read a prompt from FILE and run it.
/quit, /exit Leave the prompt.
Ctrl+C Stop generation and return to the prompt.
ds4> write a python socket example
processing 23 input tokens: 0/23 (0.0%)ds4: prompt processing failed: rocm GLM prefill failed
ds4>

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.