python / python/cpython

`concurrent.futures.Executor.map` with `buffersize` should yield from buffer and raise after executor shutdown

未關閉
#146,392 0 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

stdlib topic-multiprocessing type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

As pointed by @salomvary in this discussion, the behavior of concurrent.futures.Executor.map with buffersize is misleading when iterating after the executor's shutdown. There is 2 cases:

  1. Create the generator, shut down the executor, start iterating:
with ThreadPoolExecutor(1) as executor:
    iterator = executor.map(str, range(8), buffersize=2)

# start iterating after the shutdown
assert next(iterator) == "0"
assert next(iterator) == "1"
# raises StopIteration
next(iterator)

In this scenario the elements from the buffer are yielded, then the iteration is stopped.

  1. Create the generator, start the iteration, shut down the executor, continue the iteration
with ThreadPoolExecutor(1) as executor:
    iterator = executor.map(str, range(8), buffersize=2)
    # start iterating before the shutdown
    assert next(iterator) == "0"

# raises "RuntimeError: cannot schedule new futures after shutdown"
next(iterator)

In this scenario a RuntimeError is raised by the first post-shutdown next, leaving not-yet-yielded results in the buffer.


I would vote for a mix of both: yield from the buffer and then raise the exception.

(the fix would be a dozen rows)

CPython versions tested on:

3.14, 3.15

Operating systems tested on:

macOS, Linux, Windows

Linked PRs
  • gh-146395

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

使用 concurrent.futures.Executor.map 及其 buffersize 引數重現 issue 中的兩種 shutdown 情境。檢查 Executor.map 的迭代路徑,然後補充對在 shutdown 後例外發生之前 yield 已緩衝結果的涵蓋,並驗證現有範例的行為符合描述。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend
Issue 類型
缺陷
難度
3/5
預估耗時
1-2 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
30/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。