python / python/cpython

Some multiprocessing APIs should be possible to import or use even when multiprocessing itself isn't available

Open
#134,634 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OS-android OS-ios type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

Android doesn't support multiprocessing because it doesn't support the System V IPC API, and iOS doesn't support subprocesses at all. However, there are a couple of things that should still work on these platforms, but currently don't:

  • multiprocessing.dummy.Pool (aka multiprocessing.pool.ThreadPool) currently depends on the multiprocessing synchronization primitives, when it could use the threading equivalents instead.

  • multiprocessing.synchronize throws an exception on import if the synchronization primitives are unavailable. But this breaks some packages like joblib, which imports the primitives but doesn't always use them (joblib/joblib#825).

Chaquopy has some monkey patches to work around these issues, but it would be better if they were fixed upstream.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs
  • gh-148050

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the multiprocessing.dummy.Pool and multiprocessing.pool.ThreadPool entry points, then inspect multiprocessing.synchronize and its import-time failure on platforms without synchronization primitives. Compare the needed behavior with threading equivalents and review linked PR gh-148050; done means the supported APIs can be imported or used without unavailable multiprocessing facilities.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.