JuliaPy / JuliaPy/PyCall.jl

thread safety

Open
#882 30 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Julia
Stars
1.5k
Forks
186
PR merge metrics
No merged PRs in 30d

Description

If Julia's `Threads.nthreads() > 1`, we may want to do some more work to ensure thread safety. In particular:

* Call `PyEval_InitThreads()` in `__init__` (this is only needed for Python ≤ 3.6, and is a no-op in Python ≥ 3.7).
* Acquire the GIL in the PyObject `finalizer`, by calling `PyGILState_Ensure` (returns an [`enum`](https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Include/pystate.h#L95-L96), i.e. a `Cint` in Julia) and `PyGILState_Release`. This is because the Julia GC may be called from threads other than the main thread, so we need to ensure that we hold Python's GIL before decref-ing.

We might also expose an API to acquire the GIL, but in general I would recommend that user code should only call Python from the main thread.

See [this forum post](https://discourse.julialang.org/t/help-with-pycall-related-segfault/55998?u=stevengj) and #881 for a possible test case.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.