Dropping concurrent.futures.Executor.map result cancels pending futures

Bug report

Bug description:

Code:

from concurrent.futures import ThreadPoolExecutor

data = [
    list(range(0, 5)),
    list(range(5, 10)),
]


def _f(x):
    print(f"Processing {x}")
    return True


print("=== 1 - No consumption from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    executor.map(_f, ints)
executor.shutdown(wait=True)

print("=== 2 - Consume all values from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    futures = executor.map(_f, ints)
    results = list(futures)
executor.shutdown(wait=True)

print("=== 3 - Consume one value from the iterator ===")

executor = ThreadPoolExecutor(max_workers=1)
for ints in data:
    futures = executor.map(_f, ints)
    first = next(futures)
executor.shutdown(wait=True)

print("=== 4 - Dropping iterator cancels remaining futures ===")

executor = ThreadPoolExecutor(max_workers=1)
futures = executor.map(_f, range(0, 5))
first = next(futures)
del futures
executor.shutdown(wait=True)

Result:

=== 1 - No consumption from the iterator ===
Processing 0
Processing 1
Processing 2
Processing 3
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 2 - Consume all values from the iterator ===
Processing 0
Processing 1
Processing 2
Processing 3
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 3 - Consume one value from the iterator ===
Processing 0
Processing 1
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
=== 4 - Dropping iterator cancels remaining futures ===
Processing 0
Processing 1

The behaviour seems to be:

• If the iterator returned from map is never used (case 1), futures are not cancelled
• If the iterator returned from map is exhausted (case 2), futures are not cancelled
• If the iterator returned from map is partially consumed and then dropped (cases 3 & 4), the remaining futures are cancelled

We hit this doing a version of case 3, calling any on the iterator, which short-circuited, causing the remaining futures to not execute. This tripped us up and seems like quite a confusing behaviour that is not flagged in the docs.

It looks like this is caused by this code: https://github.com/python/cpython/blob/main/Lib/concurrent/futures/_base.py#L669-L671

Possibly related to https://github.com/python/cpython/issues/108518

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Bắt đầu bằng cách tái hiện bốn trường hợp trên CPython 3.12/Linux, sau đó kiểm tra Lib/concurrent/futures/_base.py ở các dòng 669-671 và issue liên quan #108518. Xác định hành vi dự kiến đối với các iterator map được tiêu thụ một phần, bổ sung kiểm thử hồi quy hoặc tài liệu khi thích hợp, và xác minh rằng các trường hợp thực thi được báo cáo được xử lý nhất quán.