GLM-5.3 Flash Q4_K + CUDA + --ssd-streaming fails: routed MoE unsupported types 12/12/12
- 主要言語
- C
- スター
- 22.3k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
Environment: NVIDIA H100 PCIe 80 GB (sm_90), ~235 GiB host RAM, NVIDIA driver 580.173.02, CUDA 13.0, DS4 branch glm-5.3-flash, built with make cuda CUDA_ARCH=sm_90 -j$(nproc).
Model: GLM-5.3-Flash-Q4_K.gguf, downloaded with ./download_model.sh glm53-q4.
Repro:
./ds4 \
-m /models/ds4-models/GLM-5.3-Flash-Q4_K.gguf \
--cuda \
--ssd-streaming \
-c 32768 \
-n 32 \
--temp 0 \
-p "Reply only with OK."
Failure:
ds4: CUDA backend initialized on NVIDIA H100 PCIe (sm_90) dev=0
ds4: SSD streaming initial cuda model map restricted to token embedding
...
processing 23 input tokens: 1/23 (4.3%)
ds4: glm routed moe: unsupported types 12/12/12
ds4: GLM prefill failed
Without --ssd-streaming, model parsing works but the GLM memory guard correctly refuses full residency because the model map is ~177.77 GiB on an 80 GB GPU.
This looks like a CUDA routed-MoE dispatch/type-support gap for the GLM-5.3 Q4_K layout rather than a loading or memory-planning issue.
Happy to test patches or provide more diagnostics on the H100.
コントリビューションガイド
評価
この issue はまだ評価されていません。