Support an explicit environment mapping for multiprocessing spawn

Bug report

Bug description:

"""Linux spawn/environment race. Use --no-environment-updates as a control."""

from collections import Counter
from concurrent.futures import ThreadPoolExecutor
import multiprocessing as mp
import os
import sys
import threading
import time

def child():
    pass


def update_environment(stop):
    keys = [f"SPAWN_REPRO_{i}" for i in range(256)]
    while not stop.is_set():
        for key in keys:
            os.environ[key] = "test"
        for key in keys:
            os.environ.pop(key, None)
        time.sleep(0)


def start_process(_):
    process = mp.get_context("spawn").Process(target=child)
    try:
        process.start()
    except Exception as error:
        process.close()
        return error
    return process


if __name__ == "__main__":
    print(sys.version, flush=True)
    # Start the resource tracker before racing, using only the public API.
    warmup = mp.get_context("spawn").Process(target=child)
    warmup.start()
    warmup.join()
    warmup.close()

    stop = threading.Event()
    updater = threading.Thread(target=update_environment, args=(stop,))
    if "--no-environment-updates" not in sys.argv:
        updater.start()
    try:
        with ThreadPoolExecutor(32) as executor:
            for round_number in range(20):
                # Finish all starts before joining: avoid concurrent reaping.
                results = list(executor.map(start_process, range(32)))
                counts = Counter()
                for result in results:
                    if isinstance(result, Exception):
                        counts[repr(result)] += 1
                    else:
                        result.join()
                        counts[result.exitcode] += 1
                        result.close()
                print(f"round {round_number + 1}: {dict(counts)}", flush=True)
                if counts[0] != 32:
                    raise SystemExit(1)
    finally:
        stop.set()
        if updater.ident is not None:
            updater.join()

if you run python test.py, you will get :

round 1: {0: 32}
round 2: {0: 32}
round 3: {0: 32}
round 4: {0: 31, "BrokenPipeError(32, 'Broken pipe')": 1}

and if you run with --no-environment-updates, the subprocess create successfully.

round 1: {0: 32}
round 2: {0: 32}
round 3: {0: 32}
round 4: {0: 32}
round 5: {0: 32}
round 6: {0: 32}
round 7: {0: 32}
round 8: {0: 32}
round 9: {0: 32}
round 10: {0: 32}
round 11: {0: 32}
round 12: {0: 32}
round 13: {0: 32}
round 14: {0: 32}
round 15: {0: 32}
round 16: {0: 32}
round 17: {0: 32}
round 18: {0: 32}
round 19: {0: 32}
round 20: {0: 32}

In my tests on Linux with CPython 3.14, the multiprocessing "spawn" start method follows a vfork() + execv() path. The child shares the parent’s address space until it successfully executes a new program or terminates. During this interval, other threads in the parent can continue modifying the process environment.

With another thread concurrently modifying os.environ, I observed execv() returning EFAULT, followed by the child exiting with status 255. At the Python level, this can surface as a BrokenPipeError from Process.start(). These observations are consistent with a race between the kernel reading the environment passed to execve() and another thread modifying the underlying global environ array.

The C posix_spawn() API accepts an explicit envp argument, and Python’s subprocess.Popen(env=...) similarly allows callers to supply an environment mapping. Could we consider adding an env parameter to multiprocessing.Process for the "spawn" start method, for example mp.get_context("spawn").Process(target=child, env=child_env)? This would allow callers to provide a stable environment for the child interpreter before exec, without temporarily modifying the process-global environment.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-157656

Beginne damit, den bereitgestellten Reproducer test.py mit und ohne --no-environment-updates auszuführen, und verfolge anschließend den im Bericht beschriebenen multiprocessing spawn Process.start-Pfad. Die Arbeit ist abgeschlossen, wenn spawn-Prozesse eine explizite stabile Umgebungszuordnung erhalten können und der Reproducer erfolgreich abgeschlossen wird, ohne auf Aktualisierungen der prozessglobalen Umgebung angewiesen zu sein.