mitmproxy / mitmproxy/pdoc

Infinite recursion when documenting PyO3 complex enums

Open
#868 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
2.5k
Forks
228
PR merge metrics
No merged PRs in 30d

Description

Problem Description

pdoc enters infinite recursion when documenting extension modules created with PyO3 v0.28+, when the module
contains "complex enums" (Rust sum types translated into nested Python classes). The bug is only triggered when a .pyi typing stubs file is also present. The contents of this .pyi file does not matter, it may even be empty.

PyO3 v0.27 and earlier did not exhibit this behavior.

Steps to reproduce the behavior:
  1. Build the PyO3 extension module containing "complex enums" with e.g. maturin develop. I've created a test extension module that is sufficient to reproduce the problem: pyo3-recursive-enum.
  2. Try to generate HTML docs with pdoc -o html complex_enum.
  3. RecursionError: maximum recursion depth exceeded exception is thrown.

Full backtrace:

Traceback (most recent call last):
  File "/home/raven/pyo3-recursive-enum/.venv/bin/pdoc", line 7, in <module>
    sys.exit(cli())
             ~~~^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/__main__.py", line 198, in cli
    pdoc.pdoc(
    ~~~~~~~~~^
        *opts.modules,
        ^^^^^^^^^^^^^^
        output_directory=opts.output_directory,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/__init__.py", line 548, in pdoc
    out = render.html_module(module, all_modules)
  File "/usr/lib/python3.14/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/render.py", line 106, in html_module
    return env.get_template("module.html.jinja2").render(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        module=module,
        ^^^^^^^^^^^^^^
    ...<7 lines>...
        mtime=mtime,
        ^^^^^^^^^^^^
    )
    ^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/templates/default/module.html.jinja2", line 315, in top-level template code
    {%- if loop.nextitem -%}
    ^^^^^^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/templates/default/frame.html.jinja2", line 36, in top-level template code
    {% block body %}
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/templates/default/frame.html.jinja2", line 40, in block 'body'
    <div>{% block nav %}{% endblock %}</div>
    ^^^^^^^^^^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/templates/default/module.html.jinja2", line 56, in block 'nav'
    {% block nav_members %}
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/templates/default/module.html.jinja2", line 57, in block 'nav_members'
    {% if module.members %}
    ^^^^^^^^^^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/jinja2/environment.py", line 490, in getattr
    return getattr(obj, attribute)
  File "/usr/lib/python3.14/functools.py", line 1126, in __get__
    val = self.func(instance)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc.py", line 342, in members
    doc_pyi.include_typeinfo_from_stub_files(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_pyi.py", line 140, in include_typeinfo_from_stub_files
    _prepare_module(module)
    ~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_pyi.py", line 81, in _prepare_module
    _prepare_module(member)
    ~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_pyi.py", line 81, in _prepare_module
    _prepare_module(member)
    ~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_pyi.py", line 81, in _prepare_module
    _prepare_module(member)
    ~~~~~~~~~~~~~~~^^^^^^^^
  [Previous line repeated 973 more times]
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_pyi.py", line 79, in _prepare_module
    for member in ns.members.values():
                  ^^^^^^^^^^
  File "/usr/lib/python3.14/functools.py", line 1126, in __get__
    val = self.func(instance)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc.py", line 261, in members
    for name, obj in self._member_objects.items():
                     ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/functools.py", line 1126, in __get__
    val = self.func(instance)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc.py", line 737, in _member_objects
    for name in self._var_docstrings:
                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/functools.py", line 1126, in __get__
    val = self.func(instance)
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc.py", line 621, in _var_docstrings
    for name, docstr in doc_ast.walk_tree(cls).var_docstrings.items():
                        ~~~~~~~~~~~~~~~~~^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_ast.py", line 104, in walk_tree
    return _walk_tree(parse(obj))
                      ~~~~~^^^^^
  File "/home/raven/pyo3-recursive-enum/.venv/lib/python3.14/site-packages/pdoc/doc_ast.py", line 71, in parse
    src = get_source(obj)
RecursionError: maximum recursion depth exceeded
System Information
pdoc: 16.0.0
Python: 3.14.2
Platform: Linux-6.18.7-arch1-1-x86_64-with-glibc2.43

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the linked pyo3-recursive-enum extension, an accompanying .pyi file, and pdoc -o html complex_enum. Start in pdoc/doc_pyi.py at _prepare_module, then inspect pdoc/doc.py where members triggers stub processing. Done means documenting the PyO3 v0.28+ complex-enum module completes without RecursionError, including with an empty stub file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.