Multiple bugs in `multiprocessing.managers`: pickling, nesting, etc
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 36k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
I noticed multiple bugs in multiprocessing.mangers and explained them, with fixes, in a blog post: https://zpz.github.io/blog/python-mp-manager-2/
A summary of the bugs are as follows:
- Pickling of
BaseProxyobjects: easily causes object release before unpickling happens in another process - Unable to call proxy-returning methods on a proxy that has been passed to another process. (There is an open PR https://github.com/python/cpython/pull/99598 but my solution appears simpler and cleaner.)
- "Nested proxies" are never released (also reported in this isseu https://github.com/python/cpython/issues/103136)
- A simple enhancement (not exactly a bug): server-side exceptions lose all traceback info when propagated to user (also reported here in this issue https://github.com/python/cpython/issues/78279)
In addition, this issue
- The specification of
exposedis a mess. (See https://zpz.github.io/blog/python-mp-manager-2/#exposed-methods)
My package mpservice.multiprocessing.server_process fixes these issues with some additional enhancements and simplifications. The enhancements are covered in another blog post: https://zpz.github.io/blog/python-mp-manager-3/
I hope to engage in some discussions (esp with a core developer) before working on some PR, either local fixes or something a little more holistic.
CPython versions tested on:
3.10, 3.13
Operating systems tested on:
Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the multiprocessing.managers entry points and the linked blog post, then compare the reported behaviors with the referenced PRs and issues. Reproduce the pickling, nested-proxy, exception-traceback, and exposed-method cases; the work is done only after the scope is agreed and each selected bug has regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100