NVIDIA / NVIDIA/cccl

[DISCUSSION] Alternative solution for determining compute capability at runtime

Open
#898 2 comments 0 reactions 1 assignee Claimed by @miscco View on GitHub
cub
Dominant language
C++
Stars
2.5k
Forks
487
Avg merge
2d 7h
Merged PRs (30d)
296

Description

## Current Situation

As discussed in https://github.com/NVIDIA/cub/issues/545, CUB needs to query the current device's compute capability in order to know which tuning policy to use for launching the kernel.

Currently, CUB does this by using [`cudaFuncGetAttributes`](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__EXECUTION.html#group__CUDART__EXECUTION_1g19b1696533069c03f646e2ce2beacc00) on an [`EmptyKernel`](https://github.com/NVIDIA/cub/blob/832f5c8546ca437b81f72ec5e2b76cfaaf483d8d/cub/util_device.cuh#L322-L369).

As discussed in https://github.com/NVIDIA/cub/issues/545, this runs into problems due to the nuanced relationship among the linkage of kernels, their enclosing function, and the architectures used to compile the TU. The end result is that we can end up getting a version of `EmptyKernel` with a different PTX version than we expect.

## Proposal

The goal of the machinery described above is to determine which PTX version for a given kernel will be used when it is invoked.

However, there is another way for CUB to do this.

~We could instead use [`cudaGetDeviceProperties`](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1g1bf9d625a931d657e08db2b4391170f0). The resulting [`cudaDeviceProp`](https://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp) structure has `cudaDeviceProp::major` and `cudaDeviceProp::minor` members that indicate the major/minor versions of the compute capability for the current device.~

We could instead use [`cudaDeviceGetAttribute`](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1gb22e8256592b836df9a9cc36c9db7151) and query for [cudaDevAttrComputeCapabilityMajor](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1gg49e2f8c2c0bd6fe264f2fc970912e5cd220ff111a6616ab512e229d8f2f8bf87) and [cudaDevAttrComputeCapabilityMinor](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1gg49e2f8c2c0bd6fe264f2fc970912e5cd220ff111a6616ab512e229d8f2f8bf87).

In addition, we would have to somewhere cache internal to CUB the list of architectures used to compile a particular TU (`__CUDA_ARCH_LIST__`) so we can select the closest arch to the compute capability of the current device.

## Additional Context

~It is generally recommended to avoid using `cudaGetDeviceProperties` and to instead use [`cudaDeviceGetAttribute`](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1gb22e8256592b836df9a9cc36c9db7151) to query the specific attribute of interest as `cudaGetDeviceProperties` [can be quite slow](https://developer.nvidia.com/blog/cuda-pro-tip-the-fast-way-to-query-device-properties/).~

~However, it doesn't appear there is a way to query the compute capability through `cudaDeviceGetAttribute` as the [`cudaDeviceAttr`](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g49e2f8c2c0bd6fe264f2fc970912e5cd) enum doesn't have a field for querying compute capability.~

~I don't think this will be a serious issue as we cache the result anyways.~

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.