MODEL-MM-GLM53-FLASH: the compose — glm5_next_device.cpp and the remaining nine arms
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-glm5-next-glm5-next-for-conditional-generation
Glm5NextForConditionalGeneration (GLM-5.3-Flash) has three of eleven arms on a
device queue after W9c-3a (expert GEMM), W9c-2 (KDA recurrence, MoE router
topk), and W9c-3b (KV binding). The other eight still run on the interposed CPU
queue that Glm5NextHostForward constructs at glm5_next_forward.cpp:288. O43
discloses that, and this issue owns the wave that closes it.
Scope
Create glm5_next_device.cpp following kimi_linear_device.cpp's
single-queue, device-resident pattern. Move six arms from host to device where
providers exist on both CUDA and ROCm:
- RMSNorm (
vt::RmsNorm) - Embedding gather (
vt::Embedding/vt::EmbeddingQuant) - Chunked lm_head (
vt::Matmul) - DSA k-pool indexer (
vt::Glm5NextKpoolCompress/Select— O36) - MoE combine (
vt::MoeCombine) - Dense and shared MLPs (
vt::Matmul)
Two arms stay as host-fallback islands:
- Eager MLA attention (needs W9c-1's port to
mla::ForwardMlaAttentionBlock) - mHC sites (O34 —
kDeepseekV4Mhchas no ROCm provider)
Owed
- W9c-1 (MLA attention onto
mla::ForwardMlaAttentionBlock) — REFUSED, owns
the MLA host-fallback island - O34 (
kDeepseekV4MhcROCm provider) — owns the mHC host-fallback island - O36 (k-pool device ops wired) — DISCHARGED by this wave
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with glm5_next_forward.cpp:288 and compare the requested structure with kimi_linear_device.cpp's single-queue device-resident pattern. Trace the six listed arms and their named providers, while keeping MLA attention and mHC as host-fallback islands. Done means glm5_next_device.cpp covers the six provider-backed arms and preserves those two fallbacks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100