MODEL-MM-GLM53-FLASH-CUDA: GLM-5.3-Flash has no device arm, and it is a port onto existing kernels rather than a kernel campaign
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-GLM53-FLASH-CUDA
Glm5NextForConditionalGeneration (GLM-5.3-Flash) has no device arm.
glm5_next_forward.cpp:231-238 refuses a non-CPU queue by name, and
--device cuda has never produced a token from this model. The only generation
this row has ever observed is O30's Paris. on dgx:gpu0 with --device cpu
at 195.5 s/token.
.agents/specs/glm5-next-flash.md §W9c priced that debt as a kernel campaign on
the premise that the six forward files "carry ZERO vt::Tensor". Measured on
0b4766c96, they carry 25 across 2,783 lines — 15 in glm5_next_moe.cpp
and 10 in glm5_next_kda.cpp, which also make five vt:: compute-op calls
between them. Every primitive family this model needs already has a registered
CUDA provider, and the sibling GlmMoeDsaForCausalLM already drives
mla::ForwardMlaAttentionBlock on a GPU through a seam W3 widened to admit this
row's NoPE geometry. The debt is a PORT, not a kernel campaign.
This issue tracks the port, which the rescoping splits into four waves:
- W9b — keep-quant device residency of the 101.24 GiB artifact via
dense_attn::ResidentWeight. Open question: unified memory on GB10. - W9c-1 — retire
glm5_next_attn.cpp+glm5_next_dsa.cpp's hand-rolled
MLA/DSA block (991 host lines) ontomla::ForwardMlaAttentionBlock. 11 of 45
layers. This is also the AGENTS.md §"Shared seams" parallel-path defect (O33). - W9c-2 — lift the two CPU-only refusals (
glm5_next_kda.cpp:322-325,
glm5_next_moe.cpp:222-225) so the KDA and MoE arms reach the CUDA providers
they already call. 34 of 45 layers plus every sparse block. - W9c-3 — the compose:
Dev/DBuf, a device-carryingForwardLogits, the
input.gather_logitsdispatch predicate with a residency clause, RMSNorm /
embedding / lm_head onto their ops, and the mHC decision (O34: mHC has a CUDA
kernel and no CPU registration). This wave deletes the refusal.
None is started. The rescoping itself lands no product code.
Fleet constraint measured while scoping: on thor:gpu0 (compute_cap 11.0) a
CUDA build with -DVLLM_CPP_FLASH_ATTN=ON and CUTLASS 4.5.0 configures to
CUDA FA2 compiled-arch manifest: [] — FA2's arch table is
8.0,8.6,8.7,8.9,12.0a,12.1a (cmake/CudaArchFeatures.cmake:349). MLA prefill
on this family IS FlashAttention with no fallback beneath it, so the eventual
--device cuda end-to-end test can only ever run on dgx:gpu0 (sm_121a).
Owed entries: O32 (the arm, as four waves), O33 (the parallel path), O34 (mHC's
inverted gap), O35 (two drifted anchors in this row's own briefing).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with glm5_next_forward.cpp:231-238 and the refusal sites in glm5_next_kda.cpp:322-325 and glm5_next_moe.cpp:222-225. Compare the sibling GlmMoeDsaForCausalLM path and the four W9b/W9c waves, including mla::ForwardMlaAttentionBlock and the device-carrying compose. Done means GLM-5.3-Flash generates a token with --device cuda on dgx:gpu0, with the listed device-residency and dispatch paths enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100