OpRegistered is a presence probe, not a capability probe: kMlaPrefillAttention is registered on CUDA even when the kernel throws for lack of FA2
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm
The same row that owns test_mla_attention_block, the campaign block gate where this defect surfaces (see #2571).
What
vt::OpRegistered(op, dev) is used across this tree as a capability probe.
For OpId::kMlaPrefillAttention on CUDA it is not one: the op is registered
unconditionally, while the kernel refuses at call time on a build without
the vendored FlashAttention-2.
Registration, outside every #ifdef — src/vt/cuda/cuda_mla_prefill.cu:454-460:
struct Registrar {
Registrar() {
RegisterOp(
OpId::kMlaPrefillAttention, DeviceType::kCUDA,
reinterpret_cast<void*>(static_cast<MlaPrefillAttentionFn>(&MlaPrefillAttentionCuda)));
The refusal, inside the kernel — src/vt/cuda/cuda_mla_prefill.cu:176-184:
#ifndef VLLM_CPP_FLASH_ATTN
throw std::runtime_error(
"cuda mla_prefill_attention: built without the vendored FlashAttention-2 ...
MLA prefill on sm_121 IS FlashAttention — the upstream selector has no
fallback below it — so there is nothing to degrade to.");
#else
So on a non-FA2 CUDA build, OpRegistered(kMlaPrefillAttention, kCUDA) returns
true and the first MLA prefill throws. Any caller that asks the op table
"can this device do MLA prefill" gets the wrong answer.
Why this is not hypothetical
Two separate confirmation runs on dgx:gpu0 were interpreted against the wrong
cause because of this shape. The three CUDA cases in
test_mla_attention_block throw built without the vendored FlashAttention-2
on an idle box with 116.8 GB free, and had previously been filed as CUDA
out-of-memory (#2571) because an earlier run on a loaded box threw genuinely
different strings. One symptom, two causes, and the op table could not
distinguish either.
It is also easy to reach unintentionally: VLLM_CPP_FLASH_ATTN defaults ON
(CMakeLists.txt:2377), but the define is only applied inside
if(VLLM_CPP_FLASH_ATTN AND VLLM_CPP_CUTLASS_HEADERS AND VT_FA2_ARCHS)
(:2381, applied :2421). A build with the option ON and CUTLASS headers
absent is a non-FA2 build that reports nothing unusual until the first forward.
Why it matters beyond this op
OpRegistered being a registration probe rather than a capability probe is
load-bearing elsewhere. #2564 turned on the neighbouring fact that it is
"a native-only probe that cannot see the portable reference tier". Both are the
same underlying issue: the op table answers "is something registered here",
and callers read it as "can this device serve this call".
Options, not a decision
- Register
kMlaPrefillAttentionon CUDA only under#ifdef VLLM_CPP_FLASH_ATTN,
so absence is visible to the table. Changes what an unregistered op means for
callers that currently expect it present. - Give the registry a capability/health term distinct from presence.
- Leave the table alone and document that it is a presence probe, requiring
callers that need capability to ask something else.
Option 1 is smallest but is a semantic change to a shared seam and needs the
spec/red-before/green-after path in CLAUDE.md §"Changing the rules or a
checker". Filed rather than fixed for that reason.
Not claimed
I have not surveyed every OpRegistered caller to say how many read it as a
capability probe, and I have not checked whether other backends register ops
whose kernels refuse at call time. Both are worth doing and neither is done here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the registration in src/vt/cuda/cuda_mla_prefill.cu:454-460, the refusal at lines 176-184, and the build conditions in CMakeLists.txt:2377-2421. Read CLAUDE.md's “Changing the rules or a checker” section, inspect OpRegistered callers, and run the test_mla_attention_block campaign block; done requires an agreed probe semantic, an implementation direction, and regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- ai-infra-agents, backend, build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100