mudler / mudler/vllm.cpp

dots3-note vision tower: use_bias=true still refuses, because the shared MlpGateUpMethodBase seam has no bias arm

Open
#2,616 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-MM-dots3-note-dots3-note-for-causal-lm

W6b (#2613) lifted four of the five config arms W6a deferred to it —
adapter_type = pixel_shuffle_mlp, post_norm = false, use_qk_norm = false
and is_causal = true all compute now. use_bias = true is the fifth and it
still refuses BY NAME. This issue owns it.

What use_bias turns on upstream

DotsMoEVitConfig.use_bias (vision.py:43 @ 9035151d6) is threaded into
three places:

  • nn.Linear(dim, dim*3, bias=params.bias) and nn.Linear(dim, dim, bias=params.bias) — the attention qkv and proj
    (vision_attention.py:143-144).
  • DotsSwiGLUFFN(..., bias=bias)fc1, fc2 and fc3 of every DENSE block
    (vision.py:129-134).
  • DotsSwiGLUFFN(self.hidden_size, config.moe_intermediate_size, bias=config.use_bias) — every ROUTED EXPERT's fc1/fc2/fc3
    (vision.py:158-163).

So a use_bias = true checkpoint carries 5 extra tensors per dense block and
2 + 3 * num_routed per routed block. On the released geometry that would be
1949 tensors nothing in the tree reads.

Why W6b did not lift it

  1. Nothing ships it. dots-studio/dots3-note-prev sets use_bias: false,
    and DotsMoEVitConfig's own default is False (vision.py:43). No
    published dots3-note checkpoint sets it true.
  2. The shared MLP seam has no bias arm. Both the dense block and every
    expert reach their SwiGLU through layers::MlpGateUpMethodBase
    (include/vllm/model_executor/layers/linear.h:82), whose Apply returns
    silu(gate) * up from weights alone. Lifting use_bias means either adding
    an optional bias to that base — an edit to the seam every model in the tree
    routes its MLP through, for a configuration none of them has — or writing the
    two GEMMs by hand beside it, which is the parallel path AGENTS.md's "Shared
    seams" forbids.
  3. It would land unreached. AGENTS.md's "Nothing lands dead" asks what
    production entry point reaches new code at its own merge commit. A
    use_bias arm is reachable only from a checkpoint that declares it, and the
    only such checkpoint would be a synthetic fixture written to reach it.

Refusing by name is therefore the honest state: the message says which keys the
checkpoint set and what is missing, and the text tower of such a checkpoint
still loads beside the refusal.

Closing it

Add an optional bias operand to layers::MlpGateUpMethodBase and its
unquantized members (red-first, with the existing MLP gates proving the
no-bias path is byte-identical), load the qkv/proj/fc biases, and delete the
refusal in the same change. Gate it the way W6b gates the other lifted arms: a
fixture whose use_bias is true, served end to end through
ApiServer::handle_chat_completions, against the in-test double-precision
reference.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with AGENTS.md, layers::MlpGateUpMethodBase in include/vllm/model_executor/layers/linear.h, and the bias wiring in vision.py and vision_attention.py. Trace W6b's fixture through ApiServer::handle_chat_completions and compare it with the in-test double-precision reference. Done means a true-use_bias fixture loads qkv, projection, dense-MLP, and routed-expert biases while preserving the existing no-bias path.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.