Support an explicit environment mapping for multiprocessing spawn

Bug report

Bug description:

"""Linux spawn/environment race. Use --no-environment-updates as a control."""

from collections import Counter
from concurrent.futures import ThreadPoolExecutor
import multiprocessing as mp
import os
import sys
import threading
import time

def child():
    pass


def update_environment(stop):
    keys = [f"SPAWN_REPRO_{i}" for i in range(256)]
    while not stop.is_set():
        for key in keys:
            os.environ[key] = "test"
        for key in keys:
            os.environ.pop(key, None)
        time.sleep(0)


def start_process(_):
    process = mp.get_context("spawn").Process(target=child)
    try:
        process.start()
    except Exception as error:
        process.close()
        return error
    return process


if __name__ == "__main__":
    print(sys.version, flush=True)
    # Start the resource tracker before racing, using only the public API.
    warmup = mp.get_context("spawn").Process(target=child)
    warmup.start()
    warmup.join()
    warmup.close()

    stop = threading.Event()
    updater = threading.Thread(target=update_environment, args=(stop,))
    if "--no-environment-updates" not in sys.argv:
        updater.start()
    try:
        with ThreadPoolExecutor(32) as executor:
            for round_number in range(20):
                # Finish all starts before joining: avoid concurrent reaping.
                results = list(executor.map(start_process, range(32)))
                counts = Counter()
                for result in results:
                    if isinstance(result, Exception):
                        counts[repr(result)] += 1
                    else:
                        result.join()
                        counts[result.exitcode] += 1
                        result.close()
                print(f"round {round_number + 1}: {dict(counts)}", flush=True)
                if counts[0] != 32:
                    raise SystemExit(1)
    finally:
        stop.set()
        if updater.ident is not None:
            updater.join()

if you run python test.py, you will get :

round 1: {0: 32}
round 2: {0: 32}
round 3: {0: 32}
round 4: {0: 31, "BrokenPipeError(32, 'Broken pipe')": 1}

and if you run with --no-environment-updates, the subprocess create successfully.

round 1: {0: 32}
round 2: {0: 32}
round 3: {0: 32}
round 4: {0: 32}
round 5: {0: 32}
round 6: {0: 32}
round 7: {0: 32}
round 8: {0: 32}
round 9: {0: 32}
round 10: {0: 32}
round 11: {0: 32}
round 12: {0: 32}
round 13: {0: 32}
round 14: {0: 32}
round 15: {0: 32}
round 16: {0: 32}
round 17: {0: 32}
round 18: {0: 32}
round 19: {0: 32}
round 20: {0: 32}

In my tests on Linux with CPython 3.14, the multiprocessing "spawn" start method follows a vfork() + execv() path. The child shares the parent’s address space until it successfully executes a new program or terminates. During this interval, other threads in the parent can continue modifying the process environment.

With another thread concurrently modifying os.environ, I observed execv() returning EFAULT, followed by the child exiting with status 255. At the Python level, this can surface as a BrokenPipeError from Process.start(). These observations are consistent with a race between the kernel reading the environment passed to execve() and another thread modifying the underlying global environ array.

The C posix_spawn() API accepts an explicit envp argument, and Python’s subprocess.Popen(env=...) similarly allows callers to supply an environment mapping. Could we consider adding an env parameter to multiprocessing.Process for the "spawn" start method, for example mp.get_context("spawn").Process(target=child, env=child_env)? This would allow callers to provide a stable environment for the child interpreter before exec, without temporarily modifying the process-global environment.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-157656

Commencez par exécuter le reproducer fourni test.py avec et sans --no-environment-updates, puis suivez le chemin multiprocessing spawn Process.start décrit dans le rapport. Le travail est terminé lorsque les processus spawn peuvent recevoir un mappage d’environnement stable et explicite et que le reproducer se termine avec succès sans dépendre de mises à jour de l’environnement global du processus.