InterpreterPoolExecutor workers do not inherit modifications made to sys.path before starting.
還沒有人認領這個 Issue。
- 主要語言
- Python
- 星號
- 77.2k
- 分支
- 36k
- PR 合併指標
- PR 指標待擷取
描述
Bug report
Bug description:
All existing executors will reflect modifications to sys.path in the child thread/process/whatever that gets started. However the new InterpreterPoolExecutor is not copying this behavior, leading to unexpected import errors when for example running sub-intepreters inside of pytest, as the test directory is added to the sys.path in the parent interpreter but will be lost in the sub-interpreter.
This is easily worked around by using an initializer:
pool = InterpreterPoolExecutor(
max_workers=queue.concurrency,
initializer=on_initialize_worker,
initargs=(sys.path,),
)
def on_initialize_worker(parent_sys_path: list[str]):
"""
This method is called in each worker before it begins running jobs.
It can be used to perform any necessary setup, such as loading NLTK
datasets or calling ``django.setup()``.
By default, it replaces the running job's ``sys.path`` with the workers.
"""
# Unlike all other executors, the InterpreterPoolExecutor does not
# automatically inherit the parent process's sys.path. This is a
# workaround to ensure that the worker has the same sys.path as the
# parent process or tests will fail.
sys.path = parent_sys_path
This behavior makes sense for sub-interpreters themselves, but probably not for the InterpreterPoolExecutor.
cc. @ericsnowcurrently
CPython versions tested on:
3.13, 3.14
Operating systems tested on:
Linux
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
研究方向
首先找出 CPython 中 InterpreterPoolExecutor 的實作與測試,接著比較現有 executor 如何處理 worker 啟動前父程序對 sys.path 的變更。在建立 executor 前加入一個路徑,以重現回報的匯入失敗。當 worker 不需要 initializer 即可繼承修改後的路徑,並且有涵蓋 Python 3.13 及後續版本行為的回歸測試時,即表示完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- backend
- Issue 類型
- 缺陷
- 難度
- 3/5
- 預估耗時
- 1-2 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100