LLDEnzyme with CUDA
- Dominant language
- LLVM
- Stars
- 1.7k
- Forks
- 188
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 26
Description
The LLDEnzyme linker plugin seems not to run correctly with multisource CUDA builds.
In [this example](https://github.com/BLAST-WarpX/gradient-harness/blob/de3d43d8c685fcd45d593ddbd30e6c31965b19d0/src/cuda_full_multisource/main.cu#L38) I tried running LLDEnzyme (LLVM 21) on a CUDA build using RDC, but I get the following error:
```
clang++ --offload-arch=sm_80 -fcuda-rdc -foffload-lto -flto -fplugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/ClangEnzyme-21.so -mllvm -enzyme-enable=0 -c main.cu -o main.o
clang++: warning: CUDA version is newer than the latest partially supported version 12.8 [-Wunknown-cuda-version]
clang++ --offload-arch=sm_80 -fcuda-rdc -foffload-lto -flto -fplugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/ClangEnzyme-21.so -mllvm -enzyme-enable=0 -c saxpy.cu -o saxpy.o
clang++: warning: CUDA version is newer than the latest partially supported version 12.8 [-Wunknown-cuda-version]
clang++ --offload-arch=sm_80 -fcuda-rdc --offload-new-driver -L/opt/nvidia/hpc_sdk/Linux_x86_64/25.5/cuda/12.9/lib64 -lcudart -foffload-lto -fuse-ld=lld -flto -Wl,--pass-plugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/LLDEnzyme-21.so main.o saxpy.o -o saxpy
nvlink error : Undefined reference to '_Z17__enzyme_autodiffIvJiPfS0_iS0_S0_iS0_S0_EET_PvDpT0_' in '/tmp/saxpy.nvptx64.sm_80-085730-81dfee.cubin'
nvlink error : Undefined reference to 'enzyme_dup' in '/tmp/saxpy.nvptx64.sm_80-085730-81dfee.cubin'
/pscratch/sd/m/mabate/.spack/opt/spack/linux-zen3/llvm-21.1.4-zumlrrnlovrhs6ex6v5hwtgpqc22h3i2/bin/clang-nvlink-wrapper: error: 'nvlink' failed
clang: error: nvlink command failed with exit code 1 (use -v to see invocation)
/pscratch/sd/m/mabate/.spack/opt/spack/linux-zen3/llvm-21.1.4-zumlrrnlovrhs6ex6v5hwtgpqc22h3i2/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:16: saxpy] Error 1
```
[cuda_partial_multisource](https://github.com/BLAST-WarpX/gradient-harness/blob/de3d43d8c685fcd45d593ddbd30e6c31965b19d0/src/cuda_partial_multisource/saxpy.cu#L20) is a similar example using CUDA RDC but with ClangEnzyme, and in that case Enzyme runs correctly.
Contributor guide
Research direction
Start by reproducing the failure in the linked gradient-harness src/cuda_full_multisource/main.cu and saxpy.cu examples using the shown LLDEnzyme commands, then compare it with src/cuda_partial_multisource/saxpy.cu using ClangEnzyme. Inspect the Makefile target at line 16 and the linker-wrapper output; done means the multisource CUDA RDC build links without the reported undefined references.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100