mudler / mudler/vllm.cpp

qwen4_exp: the cause of the three disagreeing token ids is still unexplained; selection flipping does not separate the forwards

Open
#2,999 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-MM-QWEN4-EXP

What is unexplained

qwen4_exp on the released UD-IQ1_S checkpoint emits
11751 13 15767 411 2029 11 1092 369 on --device cpu and
11751 13 15767 411 1928 11 628 567 on --device cuda, from the same binary,
the same weights and the same algorithm on both sides since #2612. Three of the
eight ids differ, at indices 4, 6 and 7. Nothing measured on this row explains
why those three.

Wave MOESEL-RESULT's reading (rc job
9e0864da-9b37-4309-b863-04810de0e068, evidence
docs/bench-evidence/qwen4exp-moe-selection-fwd-20260906.md) narrows this. It
does not close it.

What the reading narrowed

  1. An instrument has now read expert selection at the disagreeing forwards.
    Forwards 4, 6 and 7 are covered, with a negative control that flips nothing.
    The row's standing "no instrument has observed a disagreeing step" clause is
    discharged by measurement.
  2. The layer-0 mechanism separates at the bracketed pair. Layer 0 shows no
    selection flip, so the bimodal top-k term is not what produces the layer-0
    MoE residue there; the keep-quant grouped expert GEMM's scale-sum
    reassociation is what remains. #2552's no-flip bound lifts from 2.139e-05
    to 4.324e-05.

What it did NOT narrow, and why this issue stays open

Selection flipping does not separate the agreeing forwards from the
disagreeing ones.
Forward 5 flips 48 of 48 expert slots and its sampled id
AGREES. Forward 7 flips 48 of 48 and its id DISAGREES. Forward 4 disagrees while
flipping fewer slots (33 of 48) than agreeing forward 5. So a per-forward flip
count cannot be read as the cause of the token disagreement, in either
direction.

The next traceable step

The selection tap answers "did the chosen experts change". It does not answer
"did the change move the logit that argmax reads". Separating those needs a tap
on the routed output or the final logit row at the three disagreeing forwards,
compared against the four agreeing ones, on the same matched pair. That needs
the 68 GB artifact and a GPU.

Do not read a flip count as a cause. Do not publish a rel(sumabs) ratio for
any of it; that axis is a difference of norms and cannot rank magnitudes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading docs/bench-evidence/qwen4exp-moe-selection-fwd-20260906.md and reproducing the matched CPU/CUDA forwards described in the issue. The next traceable step is to compare a routed-output or final-logit-row tap for forwards 4, 6, and 7 against the four agreeing forwards using the 68 GB artifact on a GPU. Done means the observed trace explains which difference produces the three token-id disagreements.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.