[Bug]: construct_if_true in moe_gemm_tma_ws_launcher.inl fails to compile at C++20
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- CPU architecture: x86_64
- GPU: not required (compile-only)
- TensorRT-LLM:
main, commit4476ee3eacc1639c28f7e3e845b67ec246f2e482 - CUDA: 13.4, nvcc V13.4.59
- Host compiler: gcc 13.3.0 (Ubuntu 24.04)
- Container:
nvcr.io/nvidia/cuda-dl-base:26.08-cuda13.4-devel-ubuntu24.04
Who can help?
Build / MoE kernels.
Description
This is a second, separate C++20 blocker found while working on #19059. It is
unrelated to the functional-cast problem fixed in #19060, so I am filing it on its
own as suggested there.
Building the grouped-GEMM instantiations at C++20 fails in
cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_launcher.inl:78:
moe_gemm_tma_ws_launcher.inl(78): error: a value of type "const float **" cannot be
used to initialize an entity of type "float"
return ReturnType{std::forward<Args>(args)...};
^
detected during instantiation of
"ReturnType tensorrt_llm::_v1::kernels::cutlass_kernels_oss::construct_if_true<FLAG, ReturnType, Args...>"
The helper is:
// Constructs an object with specific arguments only if flag is true
// This forces the if constexpr branch to properly pruned be when called from in non-template functions
template <bool FLAG, class ReturnType, class... Args>
ReturnType construct_if_true(Args&&... args)
{
if constexpr (FLAG)
{
return ReturnType{std::forward<Args>(args)...};
}
else
{
return ReturnType{};
}
}
Call sites are at lines 551, 565, 582, 590 (MainloopArguments) and 610
(EpilogueScalars).
Standard dependence
Same tree, same flags, only -std changed:
| target | C++17 | C++20 |
|---|---|---|
build/.../cutlass_instantiations/gemm_grouped/100/cutlass_kernel_file_gemm_grouped_sm100_M128_BS_group0.generated.cu |
0 errors | 2 errors |
In a full --cpp_only build at C++20 (with #19060 applied so the fp8 casts no longer
interfere), this is the only file left with errors: 24 diagnostics, all from line 78,
which is the same line instantiated 24 times. The build otherwise reaches 95%.
Why C++20 comes up at all
requirements.txt:29 pins torch>=2.12.0a0, and torch 2.12 declares
CXX_STANDARD 20 in share/cmake/Torch/TorchConfig.cmake:158. At C++17 nvcc cannot
compile the torch headers at all, so raising CMAKE_CUDA_STANDARD is the only way I
found to get a build going outside the prebuilt container. Details are in #19059.
What I did not investigate
I have not traced which MainloopArguments / EpilogueScalars member receives the
const float **, so I do not know whether the right fix is in the helper, in the call
sites, or in the argument structs. I am not proposing a patch for this one, because
getting the argument mapping wrong here would silently produce incorrect results
rather than a compile error.
Reporting it so the C++20 picture is complete alongside #19059 and #19060.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_launcher.inl:78 and inspect the MainloopArguments and EpilogueScalars call sites at lines 551, 565, 582, 590, and 610. Trace which member receives the const float** during the generated grouped-GEMM C++20 instantiation, then verify the resolved behavior with the cited cutlass_instantiations target and a full --cpp_only build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, pytorch
- Domain
- backend, build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100