numba / numba/llvmlite

Build error (symbol relocation) when installing from source

Open
#522 13 comments 1 reaction 0 assignees View on GitHub
build needtriage
Dominant language
Python
Stars
2.3k
Forks
372
Avg merge
3d 2h
Merged PRs (30d)
10

Description

I've attempted to build the llvmlite from source on an Ubuntu 18.04 instance. I'm building outside of Conda, following the directions in the docs— I've built llvm 8.0 with the supplied patches, which succeeded (but only after doing `export RECIPE_DIR=/path/to/llvmlite/conda-recipes/llvmdev`— that probably ought to be in the docs).

I'm avoiding Conda because I need to deploy to virtualenv (many of them per machine), and I'm not familiar enough with Conda to figure out how to make them play together (assuming it's possible); the Conda docs seem to suggest that attempting this is brittle.

With `export LLVM_CONFIG=/path/to/my/patched/llvm8/bin/llvm-config` running either `python3 setup.py build` or `pip3 install /path/to/local/llvmlite` (the latter being the deploy path for virtualenv) will fail with:

```
g++ -shared -I/usr/lib/llvm-numba/llvm8/include -std=c++11 -fno-exceptions -fno-rtti -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-rtti -g -DHAVE_SVML -flto assembly.cpp bitcode.cpp core.cpp initfini.cpp module.cpp value.cpp executionengine.cpp transforms.cpp passmanagers.cpp targets.cpp dylib.cpp linker.cpp object_file.cpp -o libllvmlite.so -L/usr/lib/llvm-numba/llvm8/lib -flto -Wl,--exclude-libs=ALL -lLLVMFuzzMutate -lLLVMCoverage -lLLVMDlltoolDriver -lLLVMOrcJIT -lLLVMMCA -lLLVMWindowsManifest -lLLVMTextAPI -lLLVMMCJIT -lLLVMCoroutines -lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMMIRParser -lLLVMTableGen -lLLVMIntelJITEvents -lLLVMDebugInfoDWARF -lLLVMXRay -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMX86Desc -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyAsmPrinter -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyInfo -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMNVPTXAsmPrinter -lLLVMAMDGPUDisassembler -lLLVMMCDisassembler -lLLVMAMDGPUCodeGen -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMipo -lLLVMInstrumentation -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMAsmPrinter -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc -lLLVMAMDGPUInfo -lLLVMAMDGPUAsmPrinter -lLLVMAMDGPUUtils -lLLVMObjectYAML -lLLVMOptRemarks -lLLVMLibDriver -lLLVMOption -lLLVMLineEditor -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMBitReader -lLLVMCore -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -lz -lrt -ldl -lpthread -lm

/usr/bin/ld: /tmp/ccmiOUve.ltrans0.ltrans.o: relocation R_X86_64_PC32 against symbol `LLVMPY_CreateString' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
Makefile.linux:20: recipe for target 'libllvmlite.so' failed
make: *** [libllvmlite.so] Error 1
Traceback (most recent call last):
File "/home/ubuntu/sandbox/blugger/llvmlite/ffi/build.py", line 168, in
main()
File "/home/ubuntu/sandbox/blugger/llvmlite/ffi/build.py", line 158, in main
main_posix('linux', '.so')
File "/home/ubuntu/sandbox/blugger/llvmlite/ffi/build.py", line 150, in main_posix
subprocess.check_call(['make', '-f', makefile])
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-f', 'Makefile.linux']' returned non-zero exit status 2.
error: command '/usr/bin/python3' failed with exit status 1
```
If I follow the error and add `-fPIC` to the LDFLAGS of `llvmlite/ffi/Makefile.linux`, the build succeeds and passes tests.

**Is this something that should be patched or added to the docs, or am I bungling something? Is this the correct fix?**

(Even better, if >=30.0 is expected to be available PyPI shortly, that would probably moot my problem).

thanks for any help!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the source build with LLVM_CONFIG set as described, then inspect ffi/Makefile.linux and the source-build documentation, including the RECIPE_DIR setup in conda-recipes/llvmdev. Compare the failing link command with the successful manual -fPIC change. Done means the documented source installation builds and the existing tests pass without a manual Makefile edit.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.