MagicStack / MagicStack/uvloop

echoclient.py fails on recent MacOS and Linux Python releases that default to the spawn start‑method

Open
#735 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cython
Stars
11.9k
Forks
616
PR merge metrics
No merged PRs in 30d

Description

Recent Python releases on macOS and Linux use the spawn start‑method by default for new processes.

IIRC spawn starts a brand‑new interpreter and imports the target module it relies on pickling to transport the target function.

As only module‑level objects can be pickled and run_test is defined inside the if __name__ == "__main__" block, the child process tries to unpickle and look up as an attribute of the module __mp_main__. That attribute does not exist, and the unpickler raises:

Process ForkServerProcess-2:
  File "/usr/lib/python3.14/multiprocessing/process.py", line 320, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/usr/lib/python3.14/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/concurrent/futures/process.py", line 242, in _process_worker
    call_item = call_queue.get(block=True)
  File "/usr/lib/python3.14/multiprocessing/queues.py", line 120, in get
    return _ForkingPickler.loads(res)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^
AttributeError: module '__mp_main__' has no attribute 'run_test'

I am not filing a pull request because there are at least three fixes, and which one is implemented depends on the project team's preferences.

The options I see that fit the file's stated minimal changes from upstream constraint:

  1. Move run_test to the module level.
  2. As this is I/O bound, change to the ThreadPoolExecutor
  3. Force the use of fork()
import multiprocessing
multiprocessing.set_start_method('fork', force=True)

All three options seem to benchmark close to each other on Linux/MacOS. Personally I lean towards the ThreadPoolExecutor option as that is my personal default for I/O bound concurrent operations.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with echoclient.py, especially the run_test definition inside the main block and its process-pool setup. Review the three proposed approaches against the project's minimal-changes constraint, then run the client on a Python release using spawn to confirm the failure is resolved without changing its intended behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.