python / python/cpython

Max recursion during unpickling with a proxy object

Open
#133,015 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules pending stdlib type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:
Description of Problem

A wrapper class was needed to wrap an underlying object while exposing the underlying object's attributes. https://stackoverflow.com/questions/68926132/creation-of-a-class-wrapper-in-python

When creating a wrapper object for an object to be placed on a multiprocessing queue, if the wrapper object overrides the getattr method, and the object is retrieved from the queue, a max recursion depth error is observed. The expected behavior was for the wrapper object to be returned and have the underlying object exposed in the process that was handling the object.

Example Snippet
from multiprocessing import Queue

class Foo:
    """
    Any old object
    """
    def __init__(self, var: int):
        self._var = var

    @property
    def var(self) -> int:
        return self._var

class FooWrapper:
    """
    Wrapper class to expose underlying object attrs
    """
    def __init__(self, foo: Foo):
        self.foo = foo

    def __getattr__(self, name):
        return getattr(self.foo, name)

if __name__=="__main__":
    queue = Queue()
    foo = Foo(2)
    foo_wrapper = FooWrapper(foo)
    queue.put(foo_wrapper)
    message_wrapper = queue.get()  # This fails due to max recursion depth reached

>>
    return getattr(self.foo, name)
                   ^^^^^^^^
  [Previous line repeated 994 more times]
RecursionError: maximum recursion depth exceeded
CPython versions tested on:

3.11

Operating systems tested on:

Linux, Windows

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided FooWrapper and multiprocessing.Queue reproducer on Python 3.11, then trace the queue.get() unpickling path and the wrapper's getattr. Identify why attribute access recurses before the wrapped object is restored. Done means the wrapper can be queued and retrieved without RecursionError while still exposing Foo.var, with regression coverage for the reproducer.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.