pytest-dev / pytest-dev/pytest
Test hangs when using mp.Pool on Windows/macOS using CLI runner when --import-mode=importlib
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
This is a duplicate of #5429 (which does not appear to actually be resolved, or has regressed) but I come bearing more details, in particular a matrix that should help diagnose the source of the problem.
Here's a minimal example:
from multiprocessing import Pool
def times_two(x):
return x * 2
def test_foo():
with Pool(1) as pool:
assert pool.map(times_two, [1, 2, 3, 4]) == [2, 4, 6, 8]
This is using pytest 7.3.1.
Tests were performed on Github Actions ubuntu-latest, windows-latest, and macos-latest, which correspond to Ubuntu 22.04, Windows Server 2022, and macOS 12 Monterey (not Ventura). I also validated the macOS and Linux results on macOS 13 Ventura and Ubuntu 20.04 on my own machines.
All of the below were tested with Python 3.11 on Github Actions runners. On my own machines, I tested with both 3.10 and 3.11, same results.
| Platform | Runner | Result |
|---|---|---|
| Windows | pytest |
Hangs |
| macOS | pytest |
Hangs |
| Linux | pytest |
Completes |
| macOS | PyCharm | Completes |
The last row in particular suggests that the issue is likely in the test runner. Running all tests via PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py does not hang.
– Dylan
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the minimal multiprocessing example with the CLI runner on Windows and macOS using Python 3.11 and --import-mode=importlib, then compare it with Linux and the PyCharm runner. Investigate the test-runner path involved in collection or process startup; done means the example completes consistently on the affected platforms without regressing the Linux behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100