mudler / mudler/vllm.cpp

MODEL-MM-GLM53-FLASH: the compose — glm5_next_device.cpp and the remaining nine arms

Open
#3,174 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-MM-glm5-next-glm5-next-for-conditional-generation

Glm5NextForConditionalGeneration (GLM-5.3-Flash) has three of eleven arms on a
device queue after W9c-3a (expert GEMM), W9c-2 (KDA recurrence, MoE router
topk), and W9c-3b (KV binding). The other eight still run on the interposed CPU
queue that Glm5NextHostForward constructs at glm5_next_forward.cpp:288. O43
discloses that, and this issue owns the wave that closes it.

Scope

Create glm5_next_device.cpp following kimi_linear_device.cpp's
single-queue, device-resident pattern. Move six arms from host to device where
providers exist on both CUDA and ROCm:

  • RMSNorm (vt::RmsNorm)
  • Embedding gather (vt::Embedding / vt::EmbeddingQuant)
  • Chunked lm_head (vt::Matmul)
  • DSA k-pool indexer (vt::Glm5NextKpoolCompress / Select — O36)
  • MoE combine (vt::MoeCombine)
  • Dense and shared MLPs (vt::Matmul)

Two arms stay as host-fallback islands:

  • Eager MLA attention (needs W9c-1's port to mla::ForwardMlaAttentionBlock)
  • mHC sites (O34 — kDeepseekV4Mhc has no ROCm provider)

Owed

  • W9c-1 (MLA attention onto mla::ForwardMlaAttentionBlock) — REFUSED, owns
    the MLA host-fallback island
  • O34 (kDeepseekV4Mhc ROCm provider) — owns the mHC host-fallback island
  • O36 (k-pool device ops wired) — DISCHARGED by this wave

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with glm5_next_forward.cpp:288 and compare the requested structure with kimi_linear_device.cpp's single-queue device-resident pattern. Trace the six listed arms and their named providers, while keeping MLA attention and mHC as host-fallback islands. Done means glm5_next_device.cpp covers the six provider-backed arms and preserves those two fallbacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.