EnzymeAD / EnzymeAD/Enzyme

LLDEnzyme with CUDA

Open
#3,150 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
LLVM
Stars
1.7k
Forks
188
Avg merge
1d 22h
Merged PRs (30d)
26

Description

The LLDEnzyme linker plugin seems not to run correctly with multisource CUDA builds.

In [this example](https://github.com/BLAST-WarpX/gradient-harness/blob/de3d43d8c685fcd45d593ddbd30e6c31965b19d0/src/cuda_full_multisource/main.cu#L38) I tried running LLDEnzyme (LLVM 21) on a CUDA build using RDC, but I get the following error:

```
clang++ --offload-arch=sm_80 -fcuda-rdc -foffload-lto -flto -fplugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/ClangEnzyme-21.so -mllvm -enzyme-enable=0 -c main.cu -o main.o
clang++: warning: CUDA version is newer than the latest partially supported version 12.8 [-Wunknown-cuda-version]

clang++ --offload-arch=sm_80 -fcuda-rdc -foffload-lto -flto -fplugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/ClangEnzyme-21.so -mllvm -enzyme-enable=0 -c saxpy.cu -o saxpy.o
clang++: warning: CUDA version is newer than the latest partially supported version 12.8 [-Wunknown-cuda-version]

clang++ --offload-arch=sm_80 -fcuda-rdc --offload-new-driver -L/opt/nvidia/hpc_sdk/Linux_x86_64/25.5/cuda/12.9/lib64 -lcudart -foffload-lto -fuse-ld=lld -flto -Wl,--pass-plugin=/global/homes/m/mabate/src/Enzyme/build/Enzyme/LLDEnzyme-21.so main.o saxpy.o -o saxpy

nvlink error : Undefined reference to '_Z17__enzyme_autodiffIvJiPfS0_iS0_S0_iS0_S0_EET_PvDpT0_' in '/tmp/saxpy.nvptx64.sm_80-085730-81dfee.cubin'
nvlink error : Undefined reference to 'enzyme_dup' in '/tmp/saxpy.nvptx64.sm_80-085730-81dfee.cubin'
/pscratch/sd/m/mabate/.spack/opt/spack/linux-zen3/llvm-21.1.4-zumlrrnlovrhs6ex6v5hwtgpqc22h3i2/bin/clang-nvlink-wrapper: error: 'nvlink' failed
clang: error: nvlink command failed with exit code 1 (use -v to see invocation)
/pscratch/sd/m/mabate/.spack/opt/spack/linux-zen3/llvm-21.1.4-zumlrrnlovrhs6ex6v5hwtgpqc22h3i2/bin/clang-linker-wrapper: error: 'clang' failed
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:16: saxpy] Error 1
```

[cuda_partial_multisource](https://github.com/BLAST-WarpX/gradient-harness/blob/de3d43d8c685fcd45d593ddbd30e6c31965b19d0/src/cuda_partial_multisource/saxpy.cu#L20) is a similar example using CUDA RDC but with ClangEnzyme, and in that case Enzyme runs correctly.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure in the linked gradient-harness src/cuda_full_multisource/main.cu and saxpy.cu examples using the shown LLDEnzyme commands, then compare it with src/cuda_partial_multisource/saxpy.cu using ClangEnzyme. Inspect the Makefile target at line 16 and the linker-wrapper output; done means the multisource CUDA RDC build links without the reported undefined references.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.