JuliaPy / JuliaPy/PythonCall.jl

juliacall: Julia GC triggered from a python thread causes hangs

オープン
#578 コメント 3 件 リアクション 1 件 担当者 0 名 GitHub で見る
bug
主要言語
Julia
スター
1.1k
フォーク
86
平均マージ
1日 22時間
マージ済み PR(30日)
3

説明

**Affects:** JuliaCall

**Describe the bug**
I am seeing Julia tasks launched from parallel python threads lead to sporadic hangs. Short-running tasks suffer less from this. I had no luck using the tricks from #539 or [here](https://juliapy.github.io/PythonCall.jl/stable/juliacall/#Caveat:-Julia's-task-scheduler). I think the hangs are triggered by Julia GC - disabling GC with `Base.GC.enable(false)` beforehand seems to fix the hangs - and triggering Julia GC manually from a python thread immediately causes a hang.

This is all with `PYTHON_JULIACALL_HANDLE_SIGNALS` set to `yes`, in case that matters.

```python
from juliacall import Main as jl
jl_gc = jl.Base.GC.gc

import threading
t = threading.Thread(target=jl_gc)
t.start() # hangs every time (100% CPU)
```

Terminating python gives me
```
[89363] signal (15): Terminated: 15
in expression starting at none:0
jl_gc_wait_for_the_world at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gc.c:241 [inlined]
ijl_gc_collect at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gc.c:3515
gc at ./gcutils.jl:129 [inlined]
gc at ./gcutils.jl:129 [inlined]
pyjlany_call at /Users/username/.julia/packages/PythonCall/Nr75f/src/JlWrap/any.jl:45
_jl_invoke at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gf.c:0 [inlined]
ijl_apply_generic at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gf.c:3077
(pyproj) c889f3b4c8d7:pyproj username$ _pyjl_callmethod at /Users/username/.julia/packages/PythonCall/Nr75f/src/JlWrap/base.jl:73
_pyjl_callmethod at /Users/username/.julia/packages/PythonCall/Nr75f/src/JlWrap/C.jl:63
jfptr__pyjl_callmethod_9032 at /Users/username/.julia/compiled/v1.10/PythonCall/WdXsa_v1O0R.dylib (unknown line)
_jl_invoke at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gf.c:0 [inlined]
ijl_apply_generic at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jlcapi__pyjl_callmethod_9116 at /Users/username/.julia/compiled/v1.10/PythonCall/WdXsa_v1O0R.dylib (unknown line)
method_vectorcall_VARARGS at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
PyObject_Vectorcall at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
_PyEval_EvalFrameDefault at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
_PyObject_FastCallDictTstate at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
slot_tp_call at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
_PyObject_Call at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
_PyEval_EvalFrameDefault at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
method_vectorcall at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
thread_run at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
pythread_wrapper at /Users/username/miniforge3/envs/pyproj/bin/python3.12 (unknown line)
_pthread_start at /usr/lib/system/libsystem_pthread.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
__psynch_cvwait at /usr/lib/system/libsystem_kernel.dylib (unknown line)
unknown function (ip: 0x0)
Allocations: 3689147 (Pool: 3684864; Big: 4283); GC: 6
```

**Your system**
- MacOS, Apple Silicon, Julia 1.10.5
- Linux, x86_64, Julia 1.10.6
- PythonCall/juliacall 0.9.23

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start by running the supplied Python threading reproducer with JuliaCall and inspect the PythonCall JlWrap/any.jl and JlWrap/base.jl call paths shown in the stack trace. Compare behavior with Julia GC enabled and disabled; done means the threaded GC call completes without hanging on the reported environments.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
julia, python
領域
backend
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
説明が足りない
初心者へのやさしさ
30/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。