JuliaPy / JuliaPy/PythonCall.jl

Threaded test succeeds on `pytest` and fails on `tox`

Open
#539 19 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Julia
Stars
1.1k
Forks
86
Avg merge
1d 22h
Merged PRs (30d)
3

Description

**Affects:** JuliaCall

**Describe the bug**
When I run the following MWE, the code succeeds when run under `pytest` but hangs forever under `tox`, presumably because `pytest` is launching the test from the main thread? I have `JULIA_NUM_THREADS` set to `auto`.

```
from juliacall import Main as jl
import pytest
from concurrent.futures import ThreadPoolExecutor, as_completed, wait

def test_sample():
jl.seval("""
function my_sqr()
a = rand(20, 20)
Threads.@threads for ii in 1:size(a, 1)
for jj in 1:size(a, 2)
a[ii,jj] = a[ii,jj]^2
end
end
return
end
""")

pool = ThreadPoolExecutor(2)

fs = {pool.submit(jl.my_sqr._jl_call_nogil): ix for ix in range(10)}
for future in as_completed(fs):
rank = fs[future]
results = future.result()
print(results)
```
If I run
`bs = [my_sqr() for ix in range(10)]`

instead in the test, everything works fine in both `tox` and `pytest`.

**Your system**
This is on PythonCall/juliacall 0.9.22 and Julia 1.10.4, Python 3.10.14, Mac M2.

**Additional context**
Add any other context about the problem here.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the supplied MWE under both pytest and tox with the reported PythonCall, JuliaCall, Julia, and Python versions, then compare the threaded execution behavior. The issue is resolved when the ThreadPoolExecutor calls complete under tox as they do under pytest, without changing the sequential case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.