JuliaPy / JuliaPy/PythonCall.jl

Using a custom sysimage with PythonCall causes error during juliacall initialization

オープン
#436 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る
bug
主要言語
Julia
スター
1.1k
フォーク
86
平均マージ
1日 22時間
マージ済み PR(30日)
3

説明

**Affects:** Both

**Describe the bug**
I would like to use a custom sysimage of our julia package which has PythonCall.jl as a dependency. The image builds fine, and is useable with `julia --sysimage=/path/to/myimage.so`, but when I try to use it as the custom sysimage with juliacall, I hit the following initialization error:
```
PYTHON_JULIACALL_SYSIMAGE=/path/to/myimage.so python
>>> from juliacall import Main as jl
fatal: error thrown and no exception handler available.
InitError(mod=:C, error=AssertionError(msg="CTX.which == :PyCall"))
#35 at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/context.jl:144
with_gil at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/gil.jl:10 [inlined]
with_gil at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/gil.jl:9 [inlined]
init_context at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/context.jl:141
__init__ at /home/xxx/.julia/packages/PythonCall/wXfah/src/cpython/CPython.jl:21
jfptr___init___72284 at /home/xxx/.julia/sysimages/myimage.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1880 [inlined]
jl_module_run_initializer at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:75
_finish_julia_init at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/init.c:855
julia_init at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/init.c:804
ijl_init_with_image at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/jlapi.c:66 [inlined]
ijl_init_with_image at /cache/build/builder-amdci4-4/julialang/julia-release-1-dot-9/src/jlapi.c:55
unknown function (ip: 0x7f29a0a18d1c)
unknown function (ip: 0x7f29a0a18288)
unknown function (ip: 0x7f29a0a2bab9)
unknown function (ip: 0x7f29a0a2c1b2)
_PyObject_MakeTpCall at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
PyEval_EvalCode at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a1447fbd)
unknown function (ip: 0x7f29a14304f7)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
unknown function (ip: 0x7f29a13f45bc)
PyObject_CallMethodObjArgs at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
PyImport_ImportModuleLevelObject at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
_PyEval_EvalFrameDefault at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a13df561)
PyEval_EvalCode at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a145f1fc)
unknown function (ip: 0x7f29a145f195)
unknown function (ip: 0x7f29a13c6eb6)
_PyRun_InteractiveLoopObject at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a1311c71)
PyRun_AnyFileExFlags at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
unknown function (ip: 0x7f29a130e572)
Py_BytesMain at /usr/local/bin/../lib/libpython3.11.so.1.0 (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at python (unknown line)
```

I tried a sysimage that didn't have PythonCall imported or as a dependency, and juliacall initalized as expected. Maybe this isn't possible, but the Assertion error is occuring at [this line](https://github.com/JuliaPy/PythonCall.jl/blob/main/src/C/context.jl#L147), and I'm not using PyCall anywhere, so perhaps it is a bug in `init_context`?

**Steps to reproduce**

In a fresh directory, create

1) build.jl:
```julia
using Pkg

Pkg.add("PythonCall")

using PythonCall

Pkg.add("PackageCompiler")
using PackageCompiler

create_sysimage(:PythonCall, sysimage_path="/home/xxx/.julia/sysimages/myimage.so", precompile_execution_file="precompile.jl")
```

2) precompile.jl:
```julia
using PythonCall
```

call `julia --project=. build.jl` (preferably in a fresh project) and then `PYTHON_JULIACALL_SYSIMAGE=/home/xxx/.julia/sysimages/myimage.so python -c 'from juliacall import Main as jl'`

**Your system**
Please provide detailed information about your system:
- Debian 11 (Docker container on MacOS 14.2, sysimage builds on host were failing for me)
- Julia 1.10, Python 3.11, PythonCall 0.9.15, juliacall 0.9.15
-`Pkg.status()` after running build.jl:
```
Status `~/.julia/environments/v1.10/Project.toml`
[9b87118b] PackageCompiler v2.1.16
[6099a3de] PythonCall v0.9.15
```
- `juliapkg.status()`.
```
/home/xxx/julia_depot/environments/pyjuliapkg/pyjuliapkg/juliapkg.json
Julia 1.10.0 @ /home/xxx/julia/bin/julia
Packages:
: {'uuid': 'xxx', 'dev': True, 'path': '/path/to/ourpackage.jl'}
```

**Additional context**
Add any other context about the problem here.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start with the sysimage reproduction in build.jl and precompile.jl, then inspect PythonCall.jl's src/C/context.jl around the linked assertion in init_context. Run the documented julia --project=. build.jl command and the PYTHON_JULIACALL_SYSIMAGE Python import, and consider the issue resolved when juliacall initializes successfully with the custom sysimage.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
julia, python
領域
backend, tooling
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
明確に書かれている
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。