mudler / mudler/vllm.cpp

The permissive HasCuda() skip lets a device-less CUDA build read green while measuring nothing

Open
#2,603 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: MODEL-MM-QWEN4-EXP

The gap

tests/vt/test_moe_router_tie_stability.cpp (landed by #2595, wave TIEBREAK of
#2586) guards each of its three device cases with the tree's permissive idiom:

if (!HasCuda()) {
  MESSAGE("no CUDA backend registered; skipping");
  return;
}

A CUDA build on a host with no device therefore reports 4 cases | 488
assertions | 0 failed | rc 0
and reads as a pass while measuring nothing on
device. That is not hypothetical: the wave's own orin:gpu0 lease did exactly
this, and #2595's evidence file §8 records that the binary read green there with
a broken tie-break compiled in.

The tree already has the idiom that refuses instead.
tests/parity/test_qwen27n_fp8_tower_paged_engine.cpp:149 reads
VT_REQUIRE_27N_FP8_GATE and turns absence into a hard FAILURE, so a harness
that intends the run as evidence can never record the absence as a pass.

This is a tree-wide gap, not #2595's defect. grep -rl "no CUDA backend registered; skipping" tests/ finds 14 files, of which 13 are not this one, and
the broader permissive-CUDA-skip shape appears in 34 test files. #2595 is named
here only because its whole subject is the difference between a real result and
a vacuous one, so it is the change that made the gap legible.

The discriminator the records were missing, and now carry

Assertion count alone does NOT separate the three states, because two of them
read 488. Case count AND assertion count together do:

state cases assertions rc
CPU-only build (device cases #ifdef'd out) 1 488 0
CUDA build, no device (cases run, skip, return) 4 488 0
CUDA build with a device 4 4652 0

The CPU-only row is measured on the dev box at #2595's head
(380ce21d4); the 4652 row is the thor:gpu0 result #2595 records. #2595's
Gates section previously taught only the assertion count and has been corrected
to teach both.

What closing this means

Either an env gate on the model of VT_REQUIRE_27N_FP8_GATE — one name, read by
every CUDA-device test, absence becomes FAILURE when it is set — or a harness
that asserts the expected case/assertion pair for the build it configured. It is
a tree-wide change across ~34 files and needs its own row, spec and gate; it is
NOT an in-flow fix to a test-only pull request.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/vt/test_moe_router_tie_stability.cpp and the gate pattern in tests/parity/test_qwen27n_fp8_tower_paged_engine.cpp:149, then inventory the 34 permissive CUDA-skip files with the stated grep. Define the tree-wide gate or harness behavior, including the configured absence failure and the CPU-only versus device-backed case/assertion pairs. Done means the change has its own row, specification, and gate rather than being folded into #2595.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.