NVIDIA / NVIDIA/cccl

[cuda.compute]: Investigate presence of LDL/STL instructions in kernels in CUB and cuda.compute

Open
#7,978 1 comment 0 reactions 1 assignee Claimed by @NaderAlAwar View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

In cuda.compute and c.parallel, we assert that the LDL and STL instructions are not present in the compiled kernel for performance reasons. We currently have to disable these checks for certain combinations of CTK versions + gpu arch + algorithm inputs.

We need to do a deeper dive into what is causing these instructions to appear and re-enable the checks. We can assume that the C++ versions of the algorithms do not contain these instructions as they have been tuned, which should automatically disqualify tunings that do result in those instructions. In order to narrow down what is causing these instructions to appear, we should look at the parts of cuda.compute compilation process that differ from the C++ compilation process:

1. Kernel template instantiation: we could be using different template arguments in some instances than in C++ due to type erasure. In some cases, things like `const` might be missing, which might not have a huge effect but it is still good to be consistent between the two.
2. Selecting the right tuning: the process for selecting the tuning differs between C++ and c.parallel (which cuda.compute depends on) as C++ relies fully on compile-time information while c.parallel finds the tuning at run-time. We need to ensure that the same policies are being selected for the same input types.
3. Different compilers: in cuda.compute, we use numba-cuda to compile the operators, nvrtc to compile the instantiated kernels, and LTO to generate the final compiled kernel. This is different from C++ where we only use NVCC. If we can find instances where the same code is being compiled differently, we should file bug reports.

Update: it appears that these instructions can appear even in CUB, so our investigation should include that as well

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.