stubgen doesn't find c modules in namespace of python package
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
stubgen does not find c extension modules that are placed in the namespace of a "regular" python package.
More Details:
For "regular" python packages and modules stubgen is using pkgutil.walk_packages() here to find subpackages/submodules but is not checking the namespace so it can miss some.
To Reproduce
This is how I encountered the issue (there is probably a simpler way to reproduce but I'm using gdb here as an example)
(Write your steps here:)
- get gdb with python3 support (install or download a statically compiled version)
- prepare a little wrapper script to start
stubgen:
import sys
import mypy
mypy.util.check_python_version("stubgen")
options = mypy.stubgen.Options(
pyversion=sys.version_info[:2],
no_import=False,
doc_dir="",
search_path="",
interpreter="",
ignore_errors=False,
include_private=False,
output_dir="out",
modules=[],
packages=["gdb"],
files=[],
verbose=False,
quiet=False,
export_less=False
)
mypy.stubgen.generate_stubs(options)
- start gdb and then source the previously written script:
source /path/to/script.py
(these steps are necessary because the gdb python package is only accessible through the python interpreter inside GDB)
Expected Behavior
I was expecting the pyi files to contain all objects in the gdb namespace (including the c modules that gdb puts in the gdb namespace).
Actual Behavior
E.g. gdb.Architecture (and all the other c modules) are not included (can be quickly checked by running grep -r "Architecture" inside out dir) even though they are easily accessible from withing gdb: python help(gdb.Architecture).
Your Environment
I tried it in several environments before investigating, e.g.:
- Mypy version used: both
mypy 0.761andmaster(at commit 129dba468c235fbdc01f608736554061368bcff3) - Mypy configuration options from
mypy.ini(and other config files): I used a clean env without additional configs - Python version used: Python 3.8.10
- Operating system and version: Ubuntu 20.04.3 LTS x86_64
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mypy/moduleinspect.py at the pkgutil.walk_packages() logic described in the issue, then trace how stubgen discovers regular package contents. Reproduce with the provided GDB wrapper and check the generated stubs for gdb.Architecture and other C modules. Done means C extension modules exposed in the package namespace appear in the generated .pyi files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100