dots3-note vision MoE: the SOFTMAX router and the topk<2 arm are refused, because the self-normalizing combine needs a per-token divisor
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-MM-dots3-note-dots3-note-for-causal-lm
W6b (#2613) lifts the pyramid MoE ViT refusal and implements the released
vision_config's router arm exactly: router_scoring_func = "sigmoid",
capacity_factor = 2 and so topk = min(int(2), num_routed) = 2 on every one of
the 17 routed blocks. Two upstream router arms are still refused BY NAME, and
this issue owns them.
What is refused
Dots3NoteVisionRefusal (src/vllm/model_executor/models/dots3_note_vision.cpp)
returns a message naming this issue when either holds:
router_scoring_funcis not"sigmoid"— upstream's other branch is
torch.softmax(gate_logits, dim=-1, dtype=torch.float32)(vision.py:185-186
@9035151d6).- any routed block's
topk = min(int(capacity_factor), num_routed)is below 2.
Why they are not in W6b
Upstream's combine is SELF-NORMALIZING: it accumulates a per-token
aggregated_gate and divides the routed sum by aggregated_gate + 1e-9
(vision.py:207-218 @ 9035151d6). On the sigmoid arm with topk > 1 upstream
has already renormalized the routing weights to sum to router_scale
(vision.py:182-184), so that denominator is a per-tower CONSTANT and W6b folds
it into vt::MoeCombine's routed_scale — an exact fold, not an approximation.
On the other two arms the denominator is a PER-TOKEN value, and no shared op in
include/vt/ops.h expresses a per-token scale on the combine: MoeCombine's
routed_scale is one float and MulColVecF32 broadcasts over COLUMNS, not
rows. Serving them would mean either extending vt::MoeCombine with a per-token
scale tensor on both the CPU and CUDA kernels — an edit to the op DeepSeek-V2's
SACRED token-exact path routes through — or a host round-trip per routed block
to renormalize a [L, top_k] f32 buffer, which is a device sync per block on
the image path.
Neither is W6b's scope (#2613 fixes the arm at sigmoid / capacity 2), and no
published dots3-note checkpoint selects either: dots-studio/dots3-note-prev
sets router_scoring_func = "sigmoid" and capacity_factor = 2, and
DotsMoEVitConfig's own defaults are the same (vision.py:51-52).
Closing it
Extend vt::MoeCombine with an optional [T] f32 per-token divisor (CPU and
CUDA kernels, red-first), route both arms through it, and delete the refusal in
the same change so message and behaviour cannot drift. The gate is the same
consistency gate W6b uses — the in-test double-precision reference already
spells upstream's literal aggregated_gate division, so it measures the new arm
with no change.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/vllm/model_executor/models/dots3_note_vision.cpp, include/vt/ops.h, and the existing W6b consistency gate. Trace vt::MoeCombine through its CPU and CUDA kernels, then compare the test's double-precision reference with upstream vision.py:207-218. Done means both refused router arms use the per-token divisor, the CPU and CUDA paths agree, and the refusal is removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 32/100