MODEL-MM-GLM53-FLASH-CUDA: GLM-5.3-Flash's k-pool indexer has no device op, so the device arm is not a routing-only port
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-GLM53-FLASH-CUDA
.agents/specs/glm5-next-flash.md §W9c's rescoping (#2410, #2413) priced the
device arm as "a PORT, not a kernel campaign", on the finding that "Every
primitive family this model needs already has a registered CUDA provider" and
that therefore no kernel needs writing for correctness.
That is false for one family, and it is the family on the critical path of all
11 DSA layers.
GLM-5.3-Flash's indexer is a k-pool indexer, and nothing in this tree implements it on a device
Measured on e4aa7c527.
include/vt/ops.h:130-131defines exactly two indexer ops,kDsaIndexerLogits
andkDsaTopkSelect. Its own comment at:123names them "the DSA 'Lightning
Indexer' selection pair".kDeepseekV4Dsa(:295) is the same family
(:284-285). There is no pooled-selection op of any kind.git grep -l 'kpool\|index_kpool\|compress_ape\|compress_gate' src/ include/
returns 11 files, and every one is aglm5_next_*file. Zero in the shared
seam, zero undersrc/vt/, zero in any other model.mla::MlaBlockWeightscarries exactly five indexer tensors
(mla_attention.h:510-514). This model has all five pluskpool_ape
[index_kpool, head_dim]andkpool_gate[head_dim, hidden_size]
(glm5_next_dsa.h:137,:141) — the two learned weights that ARE the
pooling. They are named tensors in the published artifact,
indexer_compressor_ape.weightandindexer_compressor_gate.weight, loaded
atglm5_next_loader.cpp:378-380.
The selection is structurally different rather than a variant. The seam picks
the top index_topk tokens from per-token logits. This model pools
index_kpool = 4 consecutive valid tokens under a learned per-channel 4-way
softmax (glm5_next_dsa.cpp:168-304), picks the top
index_topk / index_kpool = 512 pools, expands them back to member token
indices, then appends the ragged visible tail raw and unscored (:307-362). The
output is 2051 wide, not 2048 (OutputWidth(), glm5_next_dsa.cpp:56-59),
and it carries -1 sentinels and duplicates, which upstream absorbs with
scatter_add_ + ne(0).
glm5_next_dsa.cpp carries zero vt::Tensor across its 534 lines.
What this costs
The 11 DSA layers cannot reach a GPU by routing alone. Either
- a k-pool indexer CUDA kernel is written (a new op family: the pooled
compression, the pool-level top-k, and the visible-tail append), or - the indexer stays on the host, and every DSA layer costs a device-to-host
round trip on every step — 11 per step, on the critical path.
Neither is priced in .agents/specs/glm5-next-flash.md today. This issue owns
the measurement and the decision.
What it also corrects
O33 states that glm5_next_attn.cpp + glm5_next_dsa.cpp are 991 host lines
reimplementing a shipped block. It is 457 lines, not 991. glm5_next_dsa.cpp
duplicates nothing, because the seam does not implement what it computes. The
parallel-path defect is glm5_next_attn.cpp alone. O33's text lands with
#2413; the correction is recorded in the row's spec under
"### W9c-1 — the MLA route, PRICED and REFUSED".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .agents/specs/glm5-next-flash.md §W9c and inspect include/vt/ops.h, glm5_next_dsa.cpp, glm5_next_dsa.h, and glm5_next_loader.cpp. Compare the host k-pool selection path with registered device operations, then measure the cost of the host round trips versus the missing device operation. Done means the spec records a supported decision and its impact on all 11 DSA layers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100