defer(MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm): the fused-MoE scale orientation fix is inert behind the BF16-only refusal
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm
Found by wave PORTQ-2 re-deriving PORT-NOW entry 46 of 5559679229..e126687a9a
(#2646). Upstream:
94e9ef0768 vllm#50137, "Don't transpose fused MoE quantization scales in
RoutedExperts.load_weights".
Deferred work with a named gate, filed so the person who lands the gate meets
it.
The gate: quantized stacked experts are refused by dtype, upstream of the
orientation test
src/vllm/model_executor/models/qwen3_5_weights.cpp:965-977 refuses any non-BF16
stacked expert tensor by name ("A quantized stacked expert layout is OWED: see
.agents/specs/moe-bf16-stacked-experts.md"), and that check runs before the
orientation branch. The bug upstream fixes — a fused quantization SCALE of shape
(2*intermediate, 1) being transposed because it carries no hidden dim — needs a
quantized fused tensor to reach the branch, and none can.
The commit's second half is unreachable for a different reason:
hidden_dim_unpadded and unpadded_hidden appear here only inside two quoted
upstream comment lines (qwen3_5_weights.cpp:890,895). There is no
padded/unpadded hidden distinction, so the ... or self.moe_config.hidden_dim
fallback has no null case to cover.
The warning for whoever lands the quantized stacked arm
This tree mirrors the PRE-commit orientation rule and cites upstream's exact
pre-commit lines in its own comments (qwen3_5_weights.cpp:890,895), implementing
them at :986-1005 with "last dim is hidden is checked FIRST".
Port the post-commit rule, not the in-tree one: transpose only when the
hidden dim is definitively on the wrong axis, and fall back to hidden_dim when
the unpadded value is unset. Mirroring what is here today would carry upstream's
bug into the new arm.
One property this tree already has and should keep: a tensor with hidden on
neither trailing dim is REFUSED BY NAME (:989-992) rather than silently
transposed.
Nothing was executed for this finding: no build, no test, no GPU.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read src/vllm/model_executor/models/qwen3_5_weights.cpp:965-1005 and .agents/specs/moe-bf16-stacked-experts.md first, then compare the cited upstream change and its post-commit orientation rule. Done means the quantized stacked-expert arm admits valid tensors, uses the definitive hidden-dimension fallback, and refuses tensors with hidden on neither trailing dimension.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100