fix(MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm): stacked expert orientation is resolved by sniffing a shape, the heuristic upstream deleted
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm
Found by wave PORTQ-4 (#2680)
re-deriving PORT-NOW entry 144 of 5559679229..e126687a9a, upstream
36f4630d89 vllm#50727
"[Bugfix][MoE] Fix fused block-scale orientation".
What is wrong
The stacked-expert loader resolves weight orientation by sniffing a runtime
shape — the exact heuristic upstream deleted, present here under different names:
src/vllm/model_executor/models/qwen3_5_weights.cpp:988
const bool gu_canonical = gu.shape[2] == hidden; // [E, 2I, H]
src/vllm/model_executor/models/qwen3_5_weights.cpp:999
const bool dn_canonical = dn.shape[1] == hidden; // [E, H, I]
The comment at :985-987 names it as a transcription of upstream's test and
tie-break, including the degenerate 2I == H case. Upstream replaced this with an
explicit is_fused_checkpoint_transposed flag threaded from the model, precisely
because a shape probe cannot distinguish a transposed checkpoint from a
coincidentally square one.
What is not portable, stated per half
The commit's scale-layout refinement ("scale" not in weight_name or quant_method == BLOCK) is inert here: this loader refuses any non-BF16 stacked expert tensor by
name at qwen3_5_weights.cpp:965-974 ("only BF16 3-D stacked routed experts are
implemented ... A quantized stacked expert layout is OWED"). A stacked checkpoint
never carries a block scale into the orientation code. That half becomes real the
day the quantized stacked layout lands.
Blast radius, stated honestly
This is a latent-ambiguity fix, not a live wrong-logits bug.
.agents/model-matrix.md:321 records both published Qwen indices
(Qwen3.6-35B-A3B, Qwen3.8-2.4T-A95B) as canonical, so no shipped checkpoint
takes the transposed branch today. And the transposed population upstream needed the
flag for is qwen3_vl_moe, which has no MoE port here — qwen3_vl.cpp:122-137
builds a dense text tower.
It is filed because a shape probe that happens to be right on today's two artifacts
is not a guarantee, and because the next stacked checkpoint is the one that finds
out.
Size
Replace the two shape probes with an explicit per-architecture declaration threaded
from the registry, and refuse rather than guess when the declaration and the
shape disagree. ~40-80 lines across qwen3_5_weights.cpp, qwen3_5_weights.h and
the registry that constructs the layout, plus a red-first test.
Verification state
Nothing was executed — no build, no test run, no GPU, no lease. Both cited lines
were printed from the tree at a700e8da6 and re-read by the wave operator.
routed_experts.py is present at 5559679229, so the heuristic is pre-pin —
which is why this tree carries it. No prior triage: git log --all --grep 36f4630d89
is empty, and .agents/model-matrix.md:321 documents the stacked arm's landing
(#740) without mentioning the orientation ambiguity.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/vllm/model_executor/models/qwen3_5_weights.cpp:965-999 and qwen3_5_weights.h, then trace the registry that constructs the layout. Add a per-architecture orientation declaration, reject mismatches instead of inferring from shapes, and create the red-first test described by the issue. Verify with the relevant build and test suite.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100