VLLM_CPP_HIP=ON configures without hipBLASLt and then fails the build 539 TUs later
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM
VLLM_CPP_HIP=ON configures successfully on a ROCm install that has hipBLAS but
no hipBLASLt, then fails the build 539 TUs later.
What happens
CMakeLists.txt:1760-1775 treats the two math libraries asymmetrically, and
deliberately so: hipBLAS is required with a FATAL_ERROR, hipBLASLt is
best-effort and its if(...) block simply does not run when the header and
library are absent.
But src/vt/rocm/rocm_matmul_hipblaslt.hip is in the unconditional ROCm source
list (CMakeLists.txt:1704, again at 1727) and includes the header with no
guard:
#include <hipblaslt/hipblaslt.h> // src/vt/rocm/rocm_matmul_hipblaslt.hip:15
So the optional detection buys nothing. Configure reports only
-- ROCm hipBLAS: /opt/rocm/lib/libhipblas.so
with no hipBLASLt line at all, prints Configuring done / Generating done, and
the build then dies with
/src/vt/rocm/rocm_matmul_hipblaslt.hip:15:10: fatal error:
'hipblaslt/hipblaslt.h' file not found
1 error generated when compiling for gfx1151.
ninja: build stopped: subcommand failed.
after 538 of 570 objects have already been compiled.
Measured on
strix:gpu0, gfx1151, ROCm 7.2.4, HIP 7.2.53211-97f5574fe2, AMD clang
22.0.0git, inside the leased rc container. rc job 8293713b-1c93-4a5b-aa0c-a06b6245d892,
log /mnt/nas_share/rc/glm53-rocm/out3/build_cli.log:601-604, configure log
/mnt/nas_share/rc/glm53-rocm/out3/cmake.log:26.
The leased worker image carries the ROCm runtime and hipBLAS but not the
hipBLASLt development package, so this is the default state of a fresh lease on
this box rather than an unusual install.
Why it is worth fixing rather than documenting
The runtime path is already opt-in and defaults OFF —
src/vt/rocm/rocm_matmul_hipblaslt.hip:253-256 gates the heuristic on
VT_ROCM_HIPBLASLT=1 and the comment records that it "aborted on gfx1201 in
lab". So a build that cannot reach the library at all still pays for it with a
hard failure, and pays late.
Two shapes would both work, and choosing between them is a ROCm-build decision
that should be made deliberately rather than by whoever hits it next:
- Make hipBLASLt required, with the same
FATAL_ERRORhipBLAS gets, so the
refusal names the missing package at configure time instead of at TU 539. - Keep it optional and make the TU match: drop it from the source list when
VLLM_CPP_HIPBLASLT_INCLUDEis unset, and compile-guard the provider
registration.
Either way the configure-time report should state which of the two ROCm math
libraries was found, so the build's compiled feature set is readable before the
build rather than inferred from a failure.
Acceptance
- On a ROCm install without hipBLASLt,
-DVLLM_CPP_HIP=ONeither refuses at
configure time naming the package, or configures and builds to completion. - The configure log states the resolved hipBLAS and hipBLASLt status either way.
- No change to a build on an install that has both.
Found while bringing up GlmMoeDsaForCausalLM on this board (#2498).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CMakeLists.txt:1704, 1727, and 1760-1775, then inspect src/vt/rocm/rocm_matmul_hipblaslt.hip:15 and 253-256. Reproduce configuration and compilation with hipBLAS available but hipBLASLt absent, using the reported logs as reference. Done means the missing-library state is reported during configuration or the build completes, while installs with both libraries remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100