conda-forge / conda-forge/llvmdev-feedstock
Help with linking a downstream executable/library against Clang and LLVM static libraries
- Dominant language
- Shell
- Stars
- 12
- Forks
- 54
- Avg merge
- 4h 15m
- Merged PRs (30d)
- 3
Description
### Solution to issue cannot be found in the documentation.
- [x] I checked the documentation.
### Issue
Hi, I am looking for help with a quite tricky scenario that involves a dependent library that needs to link against LLVM statically. I am maintaining the conda-forge feedstock of the ROOT project at https://github.com/conda-forge/root-feedstock/. The latest released version ATOW is 6.36.06. This version of ROOT is based on LLVM and clang 18.1.8 (see https://github.com/conda-forge/root-feedstock/blob/34e4ead2f91fe886d4cdb4e2136c56ac116be8d0/recipe/recipe.yaml#L5). More recently, ROOT version 6.38.00 has been released which is based on LLVM and Clang version 20.1.8.
I am now in the process of preparing the update for the conda-forge recipe, and it's failing very early during the build of ROOT, with this type of error
```
│ │ FAILED: [code=262] core/G__Core.cxx lib/Core.pcm $SRC_DIR/build-dir/core/G__Core.cxx $SRC_DIR/build-dir/lib/Core.pcm
[more stuff here...]
│ │ : CommandLine Error: Option 'W' registered more than once!
│ │ LLVM ERROR: inconsistency in registered CommandLine options
```
This is not a new type of error, and it is already known that it is due to the fact that the executable being used during this particular step of the build was linked against `libLLVM.so`, whereas ROOT (at the moment) cannot work with the shared library, but must only link against LLVM statically. Just to give further context, you can see these already existing and related issues: https://github.com/root-project/root/issues/18387, https://github.com/llvm/llvm-project/pull/135570
And to substantiate my claim, here is the linked `libLLVM.so` in the build directory of the conda-forge build
```
ldd build_artifacts/bld/rattler-build_root_base_1765544109/work/build-dir/core/rootcling_stage1/src/rootcling_stage1
linux-vdso.so.1 (0x00007f9dffe4e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f9dffe22000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9dffe1e000)
libLLVM.so.20.1 => /usr/lib64/llvm20/lib64/libLLVM.so.20.1 (0x00007f9deae00000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f9deaa00000)
libm.so.6 => /lib64/libm.so.6 (0x00007f9dffd28000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f9dffcfc000)
libc.so.6 => /lib64/libc.so.6 (0x00007f9dea80c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9dffe50000)
libffi.so.8 => /lib64/libffi.so.8 (0x00007f9df39f0000)
libedit.so.0 => /lib64/libedit.so.0 (0x00007f9df39b6000)
libz.so.1 => /lib64/libz.so.1 (0x00007f9df3993000)
libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f9df38d0000)
libxml2.so.2 => /lib64/libxml2.so.2 (0x00007f9deaca2000)
libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f9df38a3000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f9dea7d7000)
```
At this point, I wanted to see clearly that the executable (called `rootcling_stage1`) is indeed linked against `libLLVM.so` at compilation time, and here is the make verbose output
```
[496/3180] : && $BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -
fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -I$PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/root_base-6.38.00 -
fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -f
stack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -I$PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/root_base-6.38.00 -fdebug-
prefix-map=$PREFIX=/usr/local/src/conda-prefix -fno-merge-constants -fdiagnostics-color=always -Wno-implicit-fallthrough -Wno-noexcept-type -pipe -Wshadow
-Wall -W -Woverloaded-virtual -fsigned-char -fsized-deallocation -pthread -O3 -DNDEBUG -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -
Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,$PREFIX/lib -Wl,-rpath-link,$PREFIX/lib -L$PREFIX/lib -Wl,-O2 -Wl,--sort-co
mmon -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,$PREFIX/lib -Wl,-rpath-link,$
PREFIX/lib -L$PREFIX/lib -rdynamic -Wl,--dependency-file=core/rootcling_stage1/CMakeFiles/rootcling_stage1.dir/link.d core/clib/CMakeFiles/Clib.dir/src/att
ach.c.o core/clib/CMakeFiles/Clib.dir/src/detach.c.o core/clib/CMakeFiles/Clib.dir/src/getpagesize.c.o core/clib/CMakeFiles/Clib.dir/src/keys.c.o core/clib
/CMakeFiles/Clib.dir/src/mcalloc.c.o core/clib/CMakeFiles/Clib.dir/src/mfree.c.o core/clib/CMakeFiles/Clib.dir/src/mmalloc.c.o core/clib/CMakeFiles/Clib.di
r/src/mmapsup.c.o core/clib/CMakeFiles/Clib.dir/src/mmcheck.c.o core/clib/CMakeFiles/Clib.dir/src/mrealloc.c.o core/clib/CMakeFiles/Clib.dir/src/sbrksup.c.
o core/clib/CMakeFiles/Clib.dir/src/snprintf.c.o core/clib/CMakeFiles/Clib.dir/src/strlcat.c.o core/clib/CMakeFiles/Clib.dir/src/strlcpy.c.o core/clingutil
s/CMakeFiles/ClingUtils.dir/src/RStl.cxx.o core/clingutils/CMakeFiles/ClingUtils.dir/src/TClingUtils.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/BaseSele
ctionRule.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/ClassSelectionRule.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/DictSelectionReader.cxx.o core/dic
tgen/CMakeFiles/Dictgen.dir/src/LinkdefReader.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/rootcling_impl.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/Sc
anner.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/SelectionRules.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/TModuleGenerator.cxx.o core/dictgen/CMakeF
iles/Dictgen.dir/src/VariableSelectionRule.cxx.o core/dictgen/CMakeFiles/Dictgen.dir/src/XMLReader.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/s
rc/FoundationUtils.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/src/RConversionRuleParser.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/
src/RError.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/src/RLogger.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/src/StringUtils.cxx.o
core/foundation/CMakeFiles/Foundation_Stage1.dir/src/TClassEdit.cxx.o core/foundation/CMakeFiles/Foundation_Stage1.dir/src/TError.cxx.o core/rootcling_stag
e1/CMakeFiles/rootcling_stage1.dir/src/rootcling_stage1.cxx.o -o core/rootcling_stage1/src/rootcling_stage1 interpreter/llvm-project/llvm/lib/libclingInte
rpreter.a interpreter/llvm-project/llvm/lib/libclingMetaProcessor.a interpreter/llvm-project/llvm/lib/libclingUtils.a -ldl -lpthread interpreter/llvm-
project/llvm/lib/libclingInterpreter.a interpreter/llvm-project/llvm/lib/libclingUtils.a $PREFIX/lib/libclangCodeGen.a $PREFIX/lib/libclangRewriteFronte
nd.a $PREFIX/lib/libclangFrontend.a $PREFIX/lib/libclangDriver.a $PREFIX/lib/libclangParse.a $PREFIX/lib/libclangRewrite.a $PREFIX/lib/libclangSeriali
zation.a $PREFIX/lib/libclangSema.a $PREFIX/lib/libclangAnalysis.a $PREFIX/lib/libclangASTMatchers.a $PREFIX/lib/libclangEdit.a $PREFIX/lib/libclangAP
INotes.a $PREFIX/lib/libclangSupport.a $PREFIX/lib/libLLVMCoverage.a $PREFIX/lib/libLLVMLTO.a $PREFIX/lib/libLLVMExtensions.a $PREFIX/lib/libLLVMOrcJI
T.a $PREFIX/lib/libLLVMExecutionEngine.a $PREFIX/lib/libLLVMJITLink.a $PREFIX/lib/libLLVMOrcTargetProcess.a $PREFIX/lib/libLLVMIntelJITProfiling.a $PR
EFIX/lib/libLLVMOrcShared.a $PREFIX/lib/libLLVMWindowsDriver.a $PREFIX/lib/libLLVMOption.a $PREFIX/lib/libLLVMRuntimeDyld.a $PREFIX/lib/libLLVMAArch64C
odeGen.a $PREFIX/lib/libLLVMAArch64AsmParser.a $PREFIX/lib/libLLVMAArch64Disassembler.a $PREFIX/lib/libLLVMAArch64Desc.a $PREFIX/lib/libLLVMAArch64Info
.a $PREFIX/lib/libLLVMAArch64Utils.a $PREFIX/lib/libLLVMAMDGPUCodeGen.a $PREFIX/lib/libLLVMMIRParser.a $PREFIX/lib/libLLVMAMDGPUAsmParser.a $PREFIX/li
b/libLLVMAMDGPUDisassembler.a $PREFIX/lib/libLLVMAMDGPUDesc.a $PREFIX/lib/libLLVMAMDGPUInfo.a $PREFIX/lib/libLLVMAMDGPUUtils.a $PREFIX/lib/libLLVMARMCo
deGen.a $PREFIX/lib/libLLVMARMAsmParser.a $PREFIX/lib/libLLVMARMDisassembler.a $PREFIX/lib/libLLVMARMDesc.a $PREFIX/lib/libLLVMARMInfo.a $PREFIX/lib/l
ibLLVMARMUtils.a $PREFIX/lib/libLLVMAVRCodeGen.a $PREFIX/lib/libLLVMAVRAsmParser.a $PREFIX/lib/libLLVMAVRDesc.a $PREFIX/lib/libLLVMAVRDisassembler.a $
PREFIX/lib/libLLVMAVRInfo.a $PREFIX/lib/libLLVMBPFCodeGen.a $PREFIX/lib/libLLVMPasses.a $PREFIX/lib/libLLVMCoroutines.a $PREFIX/lib/libLLVMHipStdPar.a
$PREFIX/lib/libLLVMBPFAsmParser.a $PREFIX/lib/libLLVMBPFDesc.a $PREFIX/lib/libLLVMBPFDisassembler.a $PREFIX/lib/libLLVMBPFInfo.a $PREFIX/lib/libLLVMHe
xagonCodeGen.a $PREFIX/lib/libLLVMHexagonAsmParser.a $PREFIX/lib/libLLVMHexagonDisassembler.a $PREFIX/lib/libLLVMHexagonDesc.a $PREFIX/lib/libLLVMHexag
onInfo.a $PREFIX/lib/libLLVMLanaiCodeGen.a $PREFIX/lib/libLLVMLanaiAsmParser.a $PREFIX/lib/libLLVMLanaiDisassembler.a $PREFIX/lib/libLLVMLanaiDesc.a $
PREFIX/lib/libLLVMLanaiInfo.a $PREFIX/lib/libLLVMLoongArchCodeGen.a $PREFIX/lib/libLLVMLoongArchAsmParser.a $PREFIX/lib/libLLVMLoongArchDisassembler.a
$PREFIX/lib/libLLVMLoongArchDesc.a $PREFIX/lib/libLLVMLoongArchInfo.a $PREFIX/lib/libLLVMMipsCodeGen.a $PREFIX/lib/libLLVMMipsAsmParser.a $PREFIX/lib/l
ibLLVMMipsDesc.a $PREFIX/lib/libLLVMMipsDisassembler.a $PREFIX/lib/libLLVMMipsInfo.a $PREFIX/lib/libLLVMMSP430CodeGen.a $PREFIX/lib/libLLVMMSP430AsmPar
ser.a $PREFIX/lib/libLLVMMSP430Desc.a $PREFIX/lib/libLLVMMSP430Disassembler.a $PREFIX/lib/libLLVMMSP430Info.a $PREFIX/lib/libLLVMNVPTXCodeGen.a $PREFI
X/lib/libLLVMNVPTXDesc.a $PREFIX/lib/libLLVMNVPTXInfo.a $PREFIX/lib/libLLVMPowerPCCodeGen.a $PREFIX/lib/libLLVMPowerPCAsmParser.a $PREFIX/lib/libLLVMPo
werPCDesc.a $PREFIX/lib/libLLVMPowerPCDisassembler.a $PREFIX/lib/libLLVMPowerPCInfo.a $PREFIX/lib/libLLVMRISCVCodeGen.a $PREFIX/lib/libLLVMipo.a $PREF
IX/lib/libLLVMFrontendOpenMP.a $PREFIX/lib/libLLVMFrontendOffloading.a $PREFIX/lib/libLLVMFrontendAtomic.a $PREFIX/lib/libLLVMLinker.a $PREFIX/lib/libL
LVMVectorize.a $PREFIX/lib/libLLVMSandboxIR.a $PREFIX/lib/libLLVMRISCVAsmParser.a $PREFIX/lib/libLLVMRISCVDisassembler.a $PREFIX/lib/libLLVMRISCVDesc.a
$PREFIX/lib/libLLVMRISCVInfo.a $PREFIX/lib/libLLVMSparcCodeGen.a $PREFIX/lib/libLLVMSparcAsmParser.a $PREFIX/lib/libLLVMSparcDesc.a $PREFIX/lib/libLL
VMSparcDisassembler.a $PREFIX/lib/libLLVMSparcInfo.a $PREFIX/lib/libLLVMSPIRVCodeGen.a $PREFIX/lib/libLLVMSPIRVAnalysis.a $PREFIX/lib/libLLVMSPIRVDesc.
a $PREFIX/lib/libLLVMSPIRVInfo.a $PREFIX/lib/libLLVMSystemZCodeGen.a $PREFIX/lib/libLLVMSystemZAsmParser.a $PREFIX/lib/libLLVMSystemZDisassembler.a $P
REFIX/lib/libLLVMSystemZDesc.a $PREFIX/lib/libLLVMSystemZInfo.a $PREFIX/lib/libLLVMVECodeGen.a $PREFIX/lib/libLLVMVEAsmParser.a $PREFIX/lib/libLLVMVEDe
sc.a $PREFIX/lib/libLLVMVEDisassembler.a $PREFIX/lib/libLLVMVEInfo.a $PREFIX/lib/libLLVMWebAssemblyCodeGen.a $PREFIX/lib/libLLVMWebAssemblyAsmParser.a
$PREFIX/lib/libLLVMWebAssemblyDisassembler.a $PREFIX/lib/libLLVMWebAssemblyUtils.a $PREFIX/lib/libLLVMWebAssemblyDesc.a $PREFIX/lib/libLLVMWebAssemblyI
nfo.a $PREFIX/lib/libLLVMX86CodeGen.a $PREFIX/lib/libLLVMInstrumentation.a $PREFIX/lib/libLLVMCFGuard.a $PREFIX/lib/libLLVMGlobalISel.a $PREFIX/lib/li
bLLVMIRPrinter.a $PREFIX/lib/libLLVMX86AsmParser.a $PREFIX/lib/libLLVMX86Desc.a $PREFIX/lib/libLLVMX86Disassembler.a $PREFIX/lib/libLLVMX86Info.a $PRE
FIX/lib/libLLVMXCoreCodeGen.a $PREFIX/lib/libLLVMAsmPrinter.a $PREFIX/lib/libLLVMSelectionDAG.a $PREFIX/lib/libLLVMCodeGen.a $PREFIX/lib/libLLVMScalarO
pts.a $PREFIX/lib/libLLVMAggressiveInstCombine.a $PREFIX/lib/libLLVMInstCombine.a $PREFIX/lib/libLLVMTarget.a $PREFIX/lib/libLLVMCGData.a $PREFIX/lib/
libLLVMBitWriter.a $PREFIX/lib/libLLVMObjCARCOpts.a $PREFIX/lib/libLLVMTransformUtils.a $PREFIX/lib/libLLVMAnalysis.a $PREFIX/lib/libLLVMProfileData.a
$PREFIX/lib/libLLVMSymbolize.a $PREFIX/lib/libLLVMDebugInfoPDB.a $PREFIX/lib/libLLVMDebugInfoCodeView.a $PREFIX/lib/libLLVMDebugInfoMSF.a $PREFIX/lib/
libLLVMDebugInfoBTF.a $PREFIX/lib/libLLVMDebugInfoDWARF.a $PREFIX/lib/libLLVMObject.a $PREFIX/lib/libLLVMIRReader.a $PREFIX/lib/libLLVMBitReader.a $PR
EFIX/lib/libLLVMAsmParser.a $PREFIX/lib/libLLVMMCParser.a $PREFIX/lib/libLLVMTextAPI.a $PREFIX/lib/libLLVMXCoreDesc.a $PREFIX/lib/libLLVMXCoreDisassemb
ler.a $PREFIX/lib/libLLVMCodeGenTypes.a $PREFIX/lib/libLLVMMCDisassembler.a $PREFIX/lib/libLLVMXCoreInfo.a $PREFIX/lib/libLLVMMC.a $PREFIX/lib/libclan
gAST.a $PREFIX/lib/libclangLex.a $PREFIX/lib/libclangBasic.a $PREFIX/lib/libLLVM.so.20.1 $PREFIX/lib/libLLVMCore.a $PREFIX/lib/libLLVMRemarks.a $PREF
IX/lib/libLLVMBitstreamReader.a $PREFIX/lib/libLLVMBinaryFormat.a $PREFIX/lib/libLLVMTargetParser.a $PREFIX/lib/libLLVMSupport.a -ldl -lpthread $PREF
IX/lib/libLLVMDemangle.a -lrt -lm $PREFIX/lib/libz.so $PREFIX/lib/libzstd.so &&
```
Not the presence of `$PREFIX/lib/libLLVM.so.20.1`. I have spent many hours trying to trace back what `cmake` command is causing that library to be there, but to no success either. All the dependencies on `LLVM` libraries are not real `cmake` targets, so I can't print their transitive dependencies unfortunately.
Now, since this error is not new, there was already a working solution for the conda-forge recipe in the past. This is implemented at https://github.com/conda-forge/root-feedstock/blob/34e4ead2f91fe886d4cdb4e2136c56ac116be8d0/recipe/build_root.sh#L149-L152: the build forces the LLVM configuration to default to *not* linking against shared libraries. For all previous versions of ROOT and LLVM, this small patch was enough to make builds on all platforms work.
In the investigation to understand why this problem reappeared in the ROOT build, I rolled back all changes to the ROOT build system until April 2025, which was way before the adoption of LLVM20 in the ROOT source code. During this process, I kept all other things the same, that is I reverted the changes to the CMakeLists.txt files, while never touching ROOT source code and also keeping the LLVM20 baseline. This still reproduces the error above, and `libLLVM.so` is still linked into the executable.
Then I tried something else. I rolled back to the commit in the ROOT source code *right before the adoption of LLVM20*, i.e. https://github.com/root-project/root/commit/07a8adbae3cf595719c6b918295f0061ade9c294 . At this point of the commit history, the ROOT conda-forge build proceeds fine without errors. Then I moved to the merge commit that introduced the adoption of LLVM20 in the source code, i.e. https://github.com/root-project/root/commit/23a5179d30a5256a39924588c6f47dca40e662d0. With this commit, the ROOT conda-forge build now fails with the error above.
Thus, I have established that the move to LLVM20 introduced changes which somehow disregard all the efforts on the ROOT side to keep the linker away from linking against shared objects. While these lines I introduced before https://github.com/conda-forge/root-feedstock/blob/34e4ead2f91fe886d4cdb4e2136c56ac116be8d0/recipe/build_root.sh#L149-L152 are the changes on the conda-forge recipe, ROOT itself makes extensive use of `DISABLE_LLVM_LINK_LLVM_DYLIB` and similar expressions to try and ask LLVM to link against libraries statically. See for example https://github.com/root-project/root/blob/8ea9f59bd0f8308c3b9b66c56ab5fa7f69a510d2/interpreter/cling/CMakeLists.txt#L346 https://github.com/root-project/root/blob/8ea9f59bd0f8308c3b9b66c56ab5fa7f69a510d2/interpreter/CppInterOp/lib/CppInterOp/CMakeLists.txt#L110 https://github.com/root-project/root/blob/8ea9f59bd0f8308c3b9b66c56ab5fa7f69a510d2/interpreter/CMakeLists.txt#L400
In my own tests, I have also tried forcing the `LLVM_LINK_LLVM_DYLIB=OFF` compile definition at the global level of the build, but to no effect.
I have also tried a *sparse* `git-bisect` of the LLVM codebase looking for changes to the build system between 18 and 20, but so far no success (the entire diff is more than 32K commits).
So, in summary, this is a plea for help. The ROOT project cannot currently be built with LLVM20 shipped by conda-forge because something it's forcing linking against `libLLVM.so`, which is not supported. I have tried in many ways to force linking against libraries statically, so far with no success. I cannot find anything in the commit log of LLVM that directly pinpoints to a change between LLVM 18 and 20 that could explain the fact that while LLVM18 happily accepted requests such as this --> https://github.com/conda-forge/root-feedstock/blob/34e4ead2f91fe886d4cdb4e2136c56ac116be8d0/recipe/build_root.sh#L149-L152, now LLVM20 seems to disregard the same code completely. In this situation, any help will be extremely appreciated: suggestions, debugging guidance, similar past experiences that could give me insights on how to proceed, anything would be really valuable here.
Sorry for the long post and thank you for your patience!
### Installed packages
```shell
I am not really in a conda environment yet, just trying to build a recipe locally via the docker container
```
### Environment info
```shell
I am not really in a conda environment yet, just trying to build a recipe locally via the docker container
```
Contributor guide
Research direction
Start by reproducing the ROOT 6.38.00 build failure at the rootcling_stage1 link step and inspect the verbose command together with the provided ldd output. Done means explaining or correcting the reported duplicate LLVM command-line registration so this build step no longer fails while preserving the required static LLVM linking.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100