get_function_address and get_global_value_address output do not have a reference to Engine
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 372
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 10
Description
In some cases, the ``ExecutionEngine`` instance can be garbage collected prematurely which invalidates all function and global value addresses. It appears that the ``ExecutionEngine`` destructor unmaps all associated memory, which means that calls to the function segfault after destruction.
One fix would be to return a subclass of int from ``get_function_address`` and ``get_global_value_address`` that references the ``ExecutionEngine``. This will keep the engine around as long as the pointers from it are around. I think this is the most intuitive behavior in Python.
Another "fix" is to say this is expected behavior and just add a note to the documentation for these functions.
Here's a minimal test case based off example code. It works if ``enable_fix`` is set to ``True``. It also works if the ``compile_function`` function is removed and it's contents are written directly into the top level of the script (pretty atypical behavior for Python).
```python
from ctypes import CFUNCTYPE, c_double
import llvmlite.binding as llvm
# All these initializations are required for code generation!
llvm.initialize()
llvm.initialize_native_target()
llvm.initialize_native_asmprinter() # yes, even this one
llvm_ir = """
; ModuleID = "examples/ir_fpadd.py"
target triple = "unknown-unknown-unknown"
target datalayout = ""
define double @"fpadd"(double %".1", double %".2")
{
entry:
%"res" = fadd double %".1", %".2"
ret double %"res"
}
"""
def compile_function(enable_fix=False):
engine = llvm.create_mcjit_compiler(
llvm.parse_assembly(''),
llvm.Target.from_default_triple().create_target_machine()
)
mod = llvm.parse_assembly(llvm_ir)
mod.verify()
engine.add_module(mod)
engine.finalize_object()
func_ptr = engine.get_function_address('fpadd')
func = CFUNCTYPE(c_double, c_double, c_double)(func_ptr)
if enable_fix:
# add a reference to engine so it doesn't get GC'd
func.engine = engine
return func
func = compile_function()
res = func(1.0, 3.5)
print("fpadd(...) =", res)
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the get_function_address and get_global_value_address entry points and reproduce the lifetime failure using the minimal Python example in the issue. Inspect how returned addresses relate to ExecutionEngine lifetime, then add a regression test showing that calling the returned function remains valid after compile_function returns; document the expected behavior if the fix is not adopted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100