Dropping concurrent.futures.Executor.map result cancels pending futures

Bug report

Bug description:

Code:

from concurrent.futures import ThreadPoolExecutor

data = [
    list(range(0, 5)),
    list(range(5, 10)),
]


def _f(x):
    print(f"Processing {x}")
    return True


print("=== 1 - No consumption from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    executor.map(_f, ints)
executor.shutdown(wait=True)

print("=== 2 - Consume all values from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    futures = executor.map(_f, ints)
    results = list(futures)
executor.shutdown(wait=True)

print("=== 3 - Consume one value from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    futures = executor.map(_f, ints)
    first = next(futures)
executor.shutdown(wait=True)

print("=== 4 - Dropping iterator cancels remaining futures ===")

executor = ThreadPoolExecutor(max_workers=1)
futures = executor.map(_f, range(0, 5))
first = next(futures)
del futures
executor.shutdown(wait=True)

Result:

=== 1 - No consumption from the iterator ===
Processing 0
Processing 1
Processing 2
Processing 3
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 2 - Consume all values from the iterator ===
Processing 0
Processing 1
Processing 2
Processing 3
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 3 - Consume one value from the iterator ===
Processing 0
Processing 1
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 4 - Dropping iterator cancels remaining futures ===
Processing 0
Processing 1

The behaviour seems to be:

• If the iterator returned from map is never used (case 1), futures are not cancelled
• If the iterator returned from map is exhausted (case 2), futures are not cancelled
• If the iterator returned from map is partially consumed and then dropped (cases 3 & 4), the remaining futures are cancelled

We hit this doing a version of case 3, calling any on the iterator, which short-circuited, causing the remaining futures to not execute. This tripped us up and seems like quite a confusing behaviour that is not flagged in the docs.

It looks like this is caused by this code: https://github.com/python/cpython/blob/main/Lib/concurrent/futures/_base.py#L669-L671

Possibly related to https://github.com/python/cpython/issues/108518

CPython versions tested on:

3.12

Operating systems tested on:

Linux

まず CPython 3.12/Linux で4つのケースを再現し、次に Lib/concurrent/futures/_base.py の669-671行目と関連する issue #108518 を調査します。部分的に消費された map イテレータに対する意図された動作を判断し、必要に応じてリグレッションテストまたはドキュメントを追加して、報告された実行ケースが一貫して処理されることを確認します。