JuliaPy / JuliaPy/PythonCall.jl

Using a custom sysimage with PythonCall causes error during juliacall initialization

Abierto
#436 3 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Julia
Estrellas
1.1k
Forks
86
Merge medio
1 d 22 h
PR fusionados (30 d)
3

Descripción

**Affects:** Both

**Describe the bug**
I would like to use a custom sysimage of our julia package which has PythonCall.jl as a dependency. The image builds fine, and is useable with `julia --sysimage=/path/to/myimage.so`, but when I try to use it as the custom sysimage with juliacall, I hit the following initialization error:
```
PYTHON_JULIACALL_SYSIMAGE=/path/to/myimage.so python
>>> from juliacall import Main as jl
fatal: error thrown and no exception handler available.
InitError(mod=:C, error=AssertionError(msg="CTX.which == :PyCall"))
#35 at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/context.jl:144
with_gil at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/gil.jl:10 [inlined]
with_gil at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/gil.jl:9 [inlined]
init_context at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/context.jl:141
__init__ at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/CPython.jl:21
jfptr___init___72284 at /home/xxx/.julia/sysimages/myimage.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]
jl_module_run_initializer at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:75
_finish_julia_init at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/init.c:855
julia_init at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/init.c:804
ijl_init_with_image at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/jlapi.c:66 [inlined]
ijl_init_with_image at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/jlapi.c:55
unknown function (ip: 0x7f29a0a18d1c)
unknown function (ip: 0x7f29a0a18288)
unknown function (ip: 0x7f29a0a2bab9)
unknown function (ip: 0x7f29a0a2c1b2)
_PyObject_MakeTpCall at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
PyEval_EvalCode at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a1447fbd)
unknown function (ip: 0x7f29a14304f7)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
unknown function (ip: 0x7f29a13f45bc)
PyObject_CallMethodObjArgs at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
PyImport_ImportModuleLevelObject at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
PyEval_EvalCode at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a145f1fc)
unknown function (ip: 0x7f29a145f195)
unknown function (ip: 0x7f29a13c6eb6)
_PyRun_InteractiveLoopObject at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a1311c71)
PyRun_AnyFileExFlags at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a130e572)
Py_BytesMain at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)
```

I tried a sysimage that didn't have PythonCall imported or as a dependency, and juliacall initalized as expected. Maybe this isn't possible, but the Assertion error is occuring at [this line](https://github.com/JuliaPy/PythonCall.jl/blob/main/src/C/context.jl#L147), and I'm not using PyCall anywhere, so perhaps it is a bug in `init_context`?

**Steps to reproduce**

In a fresh directory, create

1) build.jl:
```julia
using Pkg

Pkg.add("PythonCall")

using PythonCall

Pkg.add("PackageCompiler")
using PackageCompiler

create_sysimage(:PythonCall, sysimage_path="/home/xxx/.julia/sysimages/myimage.so", precompile_execution_file="precompile.jl")
```

2) precompile.jl:
```julia
using PythonCall
```

call `julia --project=. build.jl` (preferably in a fresh project) and then `PYTHON_JULIACALL_SYSIMAGE=/home/xxx/.julia/sysimages/myimage.so python -c 'from juliacall import Main as jl'`

**Your system**
Please provide detailed information about your system:
- Debian 11 (Docker container on MacOS 14.2, sysimage builds on host were failing for me)
- Julia 1.10, Python 3.11, PythonCall 0.9.15, juliacall 0.9.15
-`Pkg.status()` after running build.jl:
```
Status `~/.julia/environments/v1.10/Project.toml`
[9b87118b] PackageCompiler v2.1.16
[6099a3de] PythonCall v0.9.15
```
- `juliapkg.status()`.
```
/home/xxx/julia_depot/environments/pyjuliapkg/pyjuliapkg/juliapkg.json
Julia 1.10.0 @ /home/xxx/julia/bin/julia
Packages:
: {'uuid': 'xxx', 'dev': True, 'path': '/path/to/ourpackage.jl'}
```

**Additional context**
Add any other context about the problem here.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start with the sysimage reproduction in build.jl and precompile.jl, then inspect PythonCall.jl's src/C/context.jl around the linked assertion in init_context. Run the documented julia --project=. build.jl command and the PYTHON_JULIACALL_SYSIMAGE Python import, and consider the issue resolved when juliacall initializes successfully with the custom sysimage.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
julia, python
Área
backend, tooling
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bien especificado
Aptitud para principiantes
45/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.