mudler / mudler/vllm.cpp

CUDA 13.2 on Windows rejects sampler infinity sentinels during DFlash2 W3 compilation

Open
#1,475 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

SPEC-DFLASH2 W3 first compiled its CUDA arms on the local RTX 3090 host with CUDA 13.2 and Visual Studio 2022. The SM86 build fails in src/vt/cuda/cuda_sample.cu:32 because CUDA 13.2/MSVC expands INFINITY to a cast of 1e+300, then rejects the value as not representable by float.

Exact failing configuration:

cmake -S . -B build-dflash2-01-w3-sm86 -G "Visual Studio 17 2022" -A x64 -DVLLM_CPP_CUDA=ON -DVLLM_CPP_CUDA_ARCHITECTURES=86 -DVLLM_CPP_TRITON=OFF -DVLLM_CPP_BUILD_TESTS=ON "-DCMAKE_CUDA_FLAGS=-Xcompiler=/Zc:preprocessor"
cmake --build build-dflash2-01-w3-sm86 --config Release --target test_ops_topk_values_indices --parallel 1

Observed error:

cuda_sample.cu(32): error: floating-point value does not fit in required floating-point type
constexpr float kNegInf = -((float)(1e+300));

The same expansion produces warnings at the positive-infinity sentinels used by the pivot-bracket search. This blocks the new TopKValuesIndices CUDA arm and the existing sampler translation unit on native Windows.

Owner: SPEC-DFLASH2 W3 repair flow. The repair must preserve negative-infinity semantics, add an executing regression, run the focused selector/top-k CUDA parity tests on SM86, and keep Linux behavior unchanged.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/vt/cuda/cuda_sample.cu:32 and reproduce the Windows SM86 build using the provided CMake and Visual Studio command. Trace the sampler infinity sentinels and the TopKValuesIndices CUDA arm, then add an executing regression while preserving Linux behavior. Run the focused selector/top-k CUDA parity tests on SM86 and confirm the existing sampler translation unit builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.