qwen4_exp: the cause of the three disagreeing token ids is still unexplained; selection flipping does not separate the forwards
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-QWEN4-EXP
What is unexplained
qwen4_exp on the released UD-IQ1_S checkpoint emits
11751 13 15767 411 2029 11 1092 369 on --device cpu and
11751 13 15767 411 1928 11 628 567 on --device cuda, from the same binary,
the same weights and the same algorithm on both sides since #2612. Three of the
eight ids differ, at indices 4, 6 and 7. Nothing measured on this row explains
why those three.
Wave MOESEL-RESULT's reading (rc job
9e0864da-9b37-4309-b863-04810de0e068, evidence
docs/bench-evidence/qwen4exp-moe-selection-fwd-20260906.md) narrows this. It
does not close it.
What the reading narrowed
- An instrument has now read expert selection at the disagreeing forwards.
Forwards 4, 6 and 7 are covered, with a negative control that flips nothing.
The row's standing "no instrument has observed a disagreeing step" clause is
discharged by measurement. - The layer-0 mechanism separates at the bracketed pair. Layer 0 shows no
selection flip, so the bimodal top-k term is not what produces the layer-0
MoE residue there; the keep-quant grouped expert GEMM's scale-sum
reassociation is what remains. #2552's no-flip bound lifts from2.139e-05
to4.324e-05.
What it did NOT narrow, and why this issue stays open
Selection flipping does not separate the agreeing forwards from the
disagreeing ones. Forward 5 flips 48 of 48 expert slots and its sampled id
AGREES. Forward 7 flips 48 of 48 and its id DISAGREES. Forward 4 disagrees while
flipping fewer slots (33 of 48) than agreeing forward 5. So a per-forward flip
count cannot be read as the cause of the token disagreement, in either
direction.
The next traceable step
The selection tap answers "did the chosen experts change". It does not answer
"did the change move the logit that argmax reads". Separating those needs a tap
on the routed output or the final logit row at the three disagreeing forwards,
compared against the four agreeing ones, on the same matched pair. That needs
the 68 GB artifact and a GPU.
Do not read a flip count as a cause. Do not publish a rel(sumabs) ratio for
any of it; that axis is a difference of norms and cannot rank magnitudes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading docs/bench-evidence/qwen4exp-moe-selection-fwd-20260906.md and reproducing the matched CPU/CUDA forwards described in the issue. The next traceable step is to compare a routed-output or final-logit-row tap for forwards 4, 6, and 7 against the four agreeing forwards using the 68 GB artifact on a GPU. Done means the observed trace explains which difference produces the three token-id disagreements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100