qwen4_exp QSA widens q and the output gate to f32 where vLLM keeps the bf16 model dtype

Row: MODEL-MM-QWEN4-EXP

Spec: qwen4-exp-cuda-rmsnorm-weight-dtype.md, under ## Owed.
Campaign: #1978. Found while diagnosing #2477; not fixed in that flow, for the reasons below.

What

src/vllm/model_executor/models/qwen4_exp_qsa_block.cpp:694-696 allocates

DBuf q_f32(d, DType::kF32, {T, Hq, Dh});
DBuf gate(d, DType::kF32, {T, Hq, Dh});
vt::AttnGateSplit(d.q, q_f32.t(), gate.t(), qgate.t());

vLLM reaches the same point through torch.chunk, which does not change dtype:
vllm/model_executor/models/qwen3_next.py:426-435 (vLLM origin/main 25efcfa78,
an unpinned forward reference — vLLM landed qwen4_exp on 2026-08-31, after this
row was pinned). q and gate stay at model_config.dtype, which
vllm/models/qwen4_exp/nvidia/qsa.py:188-189 forces to bf16 for this
architecture.

Why it matters, and why it is invisible

The value is already bf16-rounded by the qgate store at :691, so the widening
costs bandwidth and not precision: T * Hq * Dh * 4 bytes where upstream moves
half that, at Hq = 24, Dh = 256. AGENTS.md names this case exactly --- a
token gate cannot see a dtype that is too wide, because the tokens still match
while the path moves twice the bytes.

Why it was not fixed in the same flow

It is not a wall. That site pairs an f32 activation with the f32 gamma the GGUF
stores and passes today, so nothing refuses and no forward stops on it.

More importantly the f32 is welded into the shared op, not into this model:
src/vt/cuda/cuda_glue.cu:181 declares
AttnGateSplitKernel(float* q_out, float* gate_out, const Tin* qgate, ...), and
the op is shared with qwen3_5.cpp:5328 and :5501. Narrowing it moves a second
model's path and needs that model's gate, which is a different row's evidence.

What would close this

Give vt::AttnGateSplit an output dtype rather than a hardcoded float, narrow
both qwen4_exp buffers to hidden.dtype, and gate the change on Qwen3.5 as well
as qwen4_exp.

Start with src/vt/cuda/cuda_glue.cu:181 and trace the shared vt::AttnGateSplit callers in src/vllm/model_executor/models/qwen4_exp_qsa_block.cpp:694-696 and qwen3_5.cpp:5328, :5501. Compare the qwen4_exp path with vllm/model_executor/models/qwen3_next.py:426-435 and vllm/models/qwen4_exp/nvidia/qsa.py:188-189. Done means the operation accepts an output dtype, qwen4_exp uses hidden.dtype for both buffers, and the Qwen3.5 path is gated appropriately.