Runtime compilation should fall back to PTX rather than CUBIN
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 45
Description
Currently, if the current device is newer than the device supported in that version of CuPy/CUDA, we are falling back to the latest CC supported by NVRTC:
For build-time compilation (CUB/Thrust compiled via `nvcc`), we are adding PTX for the latest compute capability (assuming CCs are [monotonic](https://github.com/cupy/cupy/pull/7100#discussion_r993216766)): https://github.com/cupy/cupy/blob/f2a9d4bf498f80a46b4f2ebbca6734d741c4716a/install/cupy_builder/_compiler.py#L78
For runtime-compilation (kernels compiled via NVRTC), we are using maximum compute capability supported by the NVRTC: https://github.com/cupy/cupy/blob/f2a9d4bf498f80a46b4f2ebbca6734d741c4716a/cupy/cuda/compiler.py#L156
This was intended to use PTX's forward-compatibility feature:
https://docs.nvidia.com/cuda/ampere-compatibility-guide/
> For example, PTX code generated for compute capability 7.x is supported to run on compute capability 7.x or any higher revision (major or minor), including compute capability 8.x.
However, after #5097 CuPy generates CUBIN instead of PTX at runtime, so this fallback stopped working as expected. Runtime compilation should be done in PTX when fallback is needed.
This was reported on the forum: https://groups.google.com/g/cupy/c/wJdMOJa8Bxo
Contributor guide
Assessment
This issue has not been assessed yet.