Include compiler-rt in the JIT environment (with fix proposition)
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 373
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 10
Description
Hello everyone!
compiler-rt functions aren't included with llvmlite. This makes generated code that relies on them not to work.
# Reproducer
One example is the LLVM IR here: https://pastebin.com/SNgXV0Ph . It has been generated using clang 8 from the following code under Linux/x86_64: https://pastebin.com/k5tJ5NiT .
With the Python script here (https://pastebin.com/WcAy4BSv) and llvmlite 0.29.0, we get a segfault. Indeed, the `__udivti3` generated by clang (cf. https://godbolt.org/z/AbZ_92) is part of compiler-rt. As the LLVM dynamic loader can't find this function loaded within the process, it doesn't fix the `mov rXX, 0` instruction that is supposed to get the pointer to this function, and we end up with a call to 0.
This test case has been extracted from a unit case of the Miasm project (https://github.com/cea-sec/miasm/blob/master/test/arch/x86/unit/mn_div.py).
Note that this issue arises under Windows and Linux x86/64.
# Fix proposal 1
One way to fix this is to use `llvm::ExecutionEngine::addArchive`, with a pre-compiled compiler-rt/builtins static library. I made a quick'n'dirty POC here: https://github.com/aguinet/llvmlite/commit/89862c086b3120b30fa67a9aab0c8dc904d02952 . This indeed fixes the issue for Linux. I still have to try this under Windows.
# Fix proposal 2
One other fix that could work is to statically compile `compiler-rt` into the llvmlite .so/.dll binary. We can't use compiler-rt "as such" because we need to modify its CMake system a little bit to generate an archive with exported symbols. I tried an alternative of this under Linux by generating a .so file with the builtins symbols exported, and LD_PRELOAD it. This also fixes the bug. I don't if this would work on Windows though.
# Conclusion
Which fix would you prefer see implemented? Do you have other suggestions?
Thanks :)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked Python reproducer and LLVM IR, then inspect the proposed ExecutionEngine::addArchive approach and the llvmlite proof-of-concept commit. Compare the Linux and Windows behavior, including the Miasm mn_div.py test case; done means compiler-rt builtins resolve correctly without the reported segfault on both platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100