python / python/cpython

[subinterpreters] Refleaks on Windows Under Specific Conditions

未關閉
#117,936 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

OS-windows pending topic-subinterpreters type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

One of my recent PRs, gh-117662, led to one of the Windows refleak buildbots failing^1. I'm fairly confident the PR revealed an existing source of refleaks (albeit an unlikely one), rather that introducing new leaks. I have resolved the failures with gh-117913, but the underlying potential source of refleaks remains.

I've been able to reproduce the leak conditions, but only on Windows (for now?). The following conditions are necessary:

  • interpreter created using a config with check_multi_interp_extensions=False (e.g. "legacy")
  • interpreter imports test.support.os_helper
try:
    import _interpreters
except ModuleNotFoundError:
    import _xxsubinterpreters as _interpreters
config = _interpreters.new_config('legacy')  # critically, check_multi_interp_extensions=False
interpid = _interpreters.create(config)
_interpreters.exec(interpid, 'import test.support.os_helper')
_interpreters.destroy(interpid)
# leaks a bunch of objects

When the bug is triggered, around 190 objects are leaked. At first I thought this was interpreter finalization failing silently, but now I think it is something else.

I suspect the underlying problem relates to legacy extension modules. In the specific example above, I'm pretty sure the _ctypes module is leaking and ~190 is how many objects it holds, directly or indirectly. The main clue there is that, when I run ./python -v ..., the _ctypes module is never noted as destroyed, whereas all other modules are. As to test.support.os_helper, _ctypes is one of the modules that gets indirectly imported. All that said, I haven't been able to reproduce the leak if the subinterpreter above imports _ctypes instead.

Why did we only see this on Windows? The only clue I can think of is that on Windows stdlib extension modules are all turned into builtin modules, IIRC. The logic of the check_multi_interp_extensions check (via _PyImport_CheckSubinterpIncompatibleExtensionAllowed() in Python/import.c) is specific to extension modules, not builtin modules.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

使用提供的 _interpreters legacy-config 程式碼片段重現 Windows 洩漏,然後從 Python/import.c 中的 _PyImport_CheckSubinterpIncompatibleExtensionAllowed() 附近開始。使用報告中描述的詳細 import 輸出,比較 test.support.os_helper 和 _ctypes 的模組清理情況。當 reproducer 不再洩漏物件,且相關的 Windows 行為已由適當的回歸測試涵蓋時,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
operating-systems
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
38/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。