Allow `multiprocessing` Servers to return non-callables
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Feature or enhancement
Proposal:
The logic for the multiprocessing Server class is the following for retrieving values from an instance referenced by a proxy:
It works well the majority of the time, but it fails when trying to get something that doesn't immediately return a callable (it can probably be a bit more specific when it comes to what type of callable it is). A change like the following would yield more robust behavior from a base point of view and wouldn't require reimplementation in a subclass:
value_or_function = getattr(obj, methodname, SENTINEL)
if value_or_function is SENTINEL:
raise AttributeError(f"Type '{obj.__class__.__name__}' has no attribute named '{methodname}'")
if isinstance(value_of_function, typing.Callable):
try:
res = value_or_function(*args, **kwds)
except Exception as e:
msg = ('#ERROR', e)
else:
typeid = gettypeid and gettypeid.get(methodname, None)
if typeid:
rident, rexposed = self.create(conn, typeid, res)
token = Token(typeid, self.address, rident)
msg = ('#PROXY', (rexposed, token))
else:
msg = ('#RETURN', res)
else:
msg = ('#RETURN', value_or_function)
There are likely edge cases that the above does not account for, but a change similar to the one above will open up the use of instance member variables and getters for properties.
A value for SENTINEL (or better) would obviously have to be defined somewhere (it's not in the example).
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in Lib/multiprocessing/managers.py at the Server retrieval logic linked in the issue. Trace how proxy attribute access distinguishes callable values from returned values, including the missing-attribute case and proxy creation path. Done means instance member variables and property getters can be retrieved without reimplementation while existing callable behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100