mudler / mudler/vllm.cpp

port(BACKEND-CUDA-COMP-FA): the MLA prefill priority list is the pre-bf2b45b5d6 order and takes no MLA dimensions

Open
#2,640 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-CUDA-COMP-FA

Found by re-deriving upstream bf2b45b5d6 (vllm#42669) against this tree in
#2632, wave PORTQ-1.

The gap

LookupMlaPrefillPriority is a 1:1 port of upstream's
_get_mla_prefill_backend_priorities, and it returns the pre-commit order.
include/vllm/platforms/cuda_attn_priority.h:144-149:

inline std::vector<std::string> LookupMlaPrefillPriority(int major) {
  if (major == 10) {  // selector.py:66-72 (Blackwell datacenter)
    return {"FLASH_ATTN", "TRTLLM_RAGGED", "FLASHINFER", "TOKENSPEED_MLA"};
  }
  return {"FLASH_ATTN"};
}

Upstream @ bf2b45b5d6 (a forward reference past the pin 5559679229) gives that
function an mla_dimensions argument (mla/prefill/selector.py:47-80) and, on
Blackwell with the GLM dims, returns
[TRTLLM_RAGGED, FLASH_ATTN, FLASHINFER, TOKENSPEED_MLA] instead
(:170-176). The port takes no dimensions and cannot express the reorder.

Reached from src/vllm/platforms/cuda.cpp:163-165
(get_mla_prefill_backend_priority) and exercised by
tests/vllm/v1/attention/test_attn_backend_registry.cpp:98.

What is NOT in scope, and why the rest of the commit is inert

The commit has three parts. Two of them are FA4-gated and unreachable here:
fa_utils.py:134,231-255 (the requires_local_attention demotion and the widened
TMEM exception) and mla/prefill/flash_attn.py:288-307 (dropping the FA4 branch
from supports_mla_dimensions). This tree ships FA2 only, recorded at
include/vllm/v1/attention/backend.h:445-448; get_flash_attn_version and
supports_mla_dimensions have no counterpart here at all.

Honest weight

Mirror fidelity, not a behaviour change on any board present. Only
FLASH_ATTN is implemented here, and this tree's device is sm_121 (major 12),
so the major == 10 branch the commit reorders is not taken on hardware anyone
runs today. A reviewer may reasonably down-rank this. It is filed because the
divergence sits inside a function whose whole purpose is to mirror upstream's
selector, and because a stale priority list is exactly the kind of thing that is
discovered late, on the first Blackwell datacenter board.

Size

~15-25 lines: an MLA-dimensions parameter and the GLM branch in
include/vllm/platforms/cuda_attn_priority.h, threading the dims through the
virtual in src/vllm/platforms/cuda.cpp and
include/vllm/platforms/interface.h:485-489, plus one registry test case.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with LookupMlaPrefillPriority in include/vllm/platforms/cuda_attn_priority.h and trace get_mla_prefill_backend_priority through src/vllm/platforms/cuda.cpp and include/vllm/platforms/interface.h. Use tests/vllm/v1/attention/test_attn_backend_registry.cpp as the verification point; done means MLA dimensions reach the selector and the GLM Blackwell case matches the stated priority order.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.