python / python/cpython

pkgutil.walk_packages(None) imports tests and then their SkipTest exceptions ends iteration

未关闭
#131,684 6 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

pending stdlib type-bug
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

I have to be clear that in some ways this isn't technically a bug per se, it's documented in the documentation, I just think the default behavior should be different because it's not ergonomic -- and that behavior change should be documented. It's a bug in the sense of I tried to do the simplest thing possible and Python was like "NOPE" and I was like "well that's gotta be a bug in Python," and I believe I am technically wrong but a lot of people in similar positions would assume the same.

Basic problem: Stdlib contains non-importable modules, if they fail with ImportError then pkgutil.walk_packages will ignore them, and those are the only packages it will ignore by default. But stdlib contains a bunch of non-importable test-case modules that instead fail with unittest.SkipTest. This means that any use of the default args on walk_packages() errors out, you have to manually tell it that SkipTest is a fine error to ignore.

Bug description:

Steps to Reproduce

Fire up python 3.13, then run the following two lines:

>>> import pkgutil
>>> [p.name for p in pkgutil.walk_packages()]
Traceback (most recent call last):
  File "<python-input-7>", line 1, in <module>
    [p.name for p in pkgutil.walk_packages()]
                     ~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/pkgutil.py", line 93, in walk_packages
    yield from walk_packages(path, info.name+'.', onerror)
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/pkgutil.py", line 78, in walk_packages
    __import__(info.name)
    ~~~~~~~~~~^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/test/test_free_threading/__init__.py", line 8, in <module>
    raise unittest.SkipTest("GIL enabled")
unittest.case.SkipTest: GIL enabled

OR, fire up homebrew's (or poetry's) python 3.12, then run the same:

Python 3.12.9 (main, Feb  4 2025, 14:38:38) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgutil
>>> [p.name for p in pkgutil.walk_packages()]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pkgutil.py", line 93, in walk_packages
    yield from walk_packages(path, info.name+'.', onerror)
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pkgutil.py", line 78, in walk_packages
    __import__(info.name)
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/test/test_gdb/__init__.py", line 22, in <module>
    raise unittest.SkipTest("test_gdb only works on source builds at the moment.")
unittest.case.SkipTest: test_gdb only works on source builds at the moment.

Cause

Seems to be pretty obvious, these are test modules that exist all the time (whether GIL is enabled or not, whether debugger is enabled or not) but they raise an exception when they get imported.

Suggestions

pkgutil should be able to catch and discard unittest.SkipTest.

The easiest way to do this is to make unittest.SkipTest a subclass of ImportError instead of Exception, but that "smells risky" (in particular it presumably isn't only thrown when people import modules). The smarter way to do this is to just document that we by default don't error on SkipTest either and adjust the code at https://github.com/python/cpython/blob/v3.13.2/Lib/pkgutil.py#L82 accordingly.

CPython versions tested on:

3.11.11, 3.12.9, 3.13.2

Operating systems tested on:

macOS

Linked PRs
  • gh-148926

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Lib/pkgutil.py 第 82 行附近开始,针对 issue 中描述的 Python 测试包使用 pkgutil.walk_packages() 重现失败。检查链接的 PR gh-148926 以及相关文档和测试;当约定的 SkipTest 行为已实现并记录在文档中,且没有破坏现有错误处理时,即可视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
backend
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。