mudler / mudler/vllm.cpp

defer(KERNEL-ATTN-MLA-SPARSE): masked MHA for sparse MLA prefills is inert, and the in-tree routing rule is pre-fix

Open
#2,666 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: KERNEL-ATTN-MLA-SPARSE

Found by wave PORTQ-2 re-deriving PORT-NOW entries 44 and 64 of
5559679229..e126687a9a (#2646).
Upstream: 82ae4164ee vllm#48770, "[2/N] Enable masked MHA for sparse MLA
prefills", and 199644d410 vllm#50906, "Guard sparse MLA masked MHA workspace".

Deferred work with a named gate, filed so the person who lands the gate meets
it.

The gate: no sparse-MLA prefill backend is registered

_use_masked_mha keys its threshold table on the backend names
FLASHMLA_SPARSE and FLASHINFER_MLA_SPARSE. Those strings exist here only in
the priority TABLE (include/vllm/platforms/cuda_attn_priority.h:82,131,133); no
backend registers under either. include/vllm/v1/attention/registry.h:82 states
it outright: "Today the sparse entry is unregistered".

Second, independent gate: _is_masked_mha_available requires device-capability
family 100 and FA version 4. The development part is sm_121 (family 120), and
.agents/kernel-matrix.md:140 already records family-100 as unreachable on
sm_121.

Consistent with that, the workspace 199644d410 guards does not exist:
GLOBAL_TOPK_MASK, masked_mha, masked_mha_workspace_fits, topk_mask and
mask_workspace all return zero hits in src/, include/ and tests/.

The warning for whoever registers a sparse backend

This tree ports the PRE-commit routing rule and cites it verbatim:
src/vllm/model_executor/layers/attention/mla_attention.cpp:477 carries
use_dense_mha = prefill_max_seq_len <= self.topk_tokens
(sparse_mla_attention.py:296-299 @ bc2d63e650), consumed at :493 as
sparse_step.

When this row registers a sparse-MLA prefill backend, port the post-commit
form, not the one currently in-tree
: 82ae4164ee's TP and seq-len threshold
table, force-MQA override, narrowed quant-output gate and context_lens field,
and 199644d410's 128 MiB bound plus the masked_mha_workspace_fits conjunct.
Landing the pre-commit rule and the guard separately would reintroduce the
unbounded workspace the second commit exists to fix.

Nothing was executed for this finding: no build, no test, no GPU.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with include/vllm/v1/attention/registry.h:82 and the priority entries in include/vllm/platforms/cuda_attn_priority.h, then inspect mla_attention.cpp:477 and :493. The work is not ready until a sparse-MLA prefill backend is registered; done means porting the post-commit routing thresholds and workspace guards from the two named upstream commits together, including the 128 MiB bound and masked_mha_workspace_fits check.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.