python / python/cpython

pkgutil.walk_packages(None) imports tests and then their SkipTest exceptions ends iteration

オープン
#131,684 コメント 6 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

pending stdlib type-bug
主要言語
Python
スター
77.2k
フォーク
35.9k
PR マージ指標
PR 指標を取得中

説明

Bug report

I have to be clear that in some ways this isn't technically a bug per se, it's documented in the documentation, I just think the default behavior should be different because it's not ergonomic -- and that behavior change should be documented. It's a bug in the sense of I tried to do the simplest thing possible and Python was like "NOPE" and I was like "well that's gotta be a bug in Python," and I believe I am technically wrong but a lot of people in similar positions would assume the same.

Basic problem: Stdlib contains non-importable modules, if they fail with ImportError then pkgutil.walk_packages will ignore them, and those are the only packages it will ignore by default. But stdlib contains a bunch of non-importable test-case modules that instead fail with unittest.SkipTest. This means that any use of the default args on walk_packages() errors out, you have to manually tell it that SkipTest is a fine error to ignore.

Bug description:

Steps to Reproduce

Fire up python 3.13, then run the following two lines:

>>> import pkgutil
>>> [p.name for p in pkgutil.walk_packages()]
Traceback (most recent call last):
  File "<python-input-7>", line 1, in <module>
    [p.name for p in pkgutil.walk_packages()]
                     ~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/pkgutil.py", line 93, in walk_packages
    yield from walk_packages(path, info.name+'.', onerror)
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/pkgutil.py", line 78, in walk_packages
    __import__(info.name)
    ~~~~~~~~~~^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.2/Frameworks/Python.framework/Versions/3.13/lib/python3.13/test/test_free_threading/__init__.py", line 8, in <module>
    raise unittest.SkipTest("GIL enabled")
unittest.case.SkipTest: GIL enabled

OR, fire up homebrew's (or poetry's) python 3.12, then run the same:

Python 3.12.9 (main, Feb  4 2025, 14:38:38) [Clang 16.0.0 (clang-1600.0.26.6)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgutil
>>> [p.name for p in pkgutil.walk_packages()]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pkgutil.py", line 93, in walk_packages
    yield from walk_packages(path, info.name+'.', onerror)
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pkgutil.py", line 78, in walk_packages
    __import__(info.name)
  File "/opt/homebrew/Cellar/python@3.12/3.12.9/Frameworks/Python.framework/Versions/3.12/lib/python3.12/test/test_gdb/__init__.py", line 22, in <module>
    raise unittest.SkipTest("test_gdb only works on source builds at the moment.")
unittest.case.SkipTest: test_gdb only works on source builds at the moment.

Cause

Seems to be pretty obvious, these are test modules that exist all the time (whether GIL is enabled or not, whether debugger is enabled or not) but they raise an exception when they get imported.

Suggestions

pkgutil should be able to catch and discard unittest.SkipTest.

The easiest way to do this is to make unittest.SkipTest a subclass of ImportError instead of Exception, but that "smells risky" (in particular it presumably isn't only thrown when people import modules). The smarter way to do this is to just document that we by default don't error on SkipTest either and adjust the code at https://github.com/python/cpython/blob/v3.13.2/Lib/pkgutil.py#L82 accordingly.

CPython versions tested on:

3.11.11, 3.12.9, 3.13.2

Operating systems tested on:

macOS

Linked PRs
  • gh-148926

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

Lib/pkgutil.py の 82 行目付近から始め、issue で説明されている Python テストパッケージに対して pkgutil.walk_packages() を使用して失敗を再現します。リンクされている PR gh-148926 と、その周辺のドキュメントおよびテストを確認します。既存のエラー処理を壊さずに、合意された SkipTest の動作が実装され、ドキュメント化されていれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
backend
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
25/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。