JuliaPy / JuliaPy/PythonCall.jl
Segfaults with frequent python calls from thread 1 on multithreaded Julia
- Dominant language
- Julia
- Stars
- 1.1k
- Forks
- 86
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 3
Description
In a complex application with high-frequency python calls I always run into
```
signal (11): Segmentation fault
in expression starting at REPL[41]:2
_PyTrash_begin at /opt/conda/conda-bld/python-split_1649141344976/work/Objects/object.c:2125
```
segfaults when Julia has multiple threads (e.g. `JULIA_NUM_THREADS=4`). I do make sure to only use PythonCall from thread 1 (and I use asserts to verify that) but somehow that doesn't seem to be enough. When running with `JULIA_NUM_THREADS=1`, all is fine. Is PythonCall incompatible with multi-threaded Julia in general?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reducing the reported high-frequency PythonCall workload and compare it with JULIA_NUM_THREADS=1 and JULIA_NUM_THREADS=4, while retaining the thread-1 assertion. Use the reported _PyTrash_begin segmentation fault as the initial failure to investigate. Done means establishing whether multithreaded Julia is generally incompatible here or identifying a reproducible condition and its fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100