Docs: `set_forkserver_preload()` does not cover lazily imported modules

Title: Docs: set_forkserver_preload() does not cover lazily imported modules

Category: Documentation (misleading documentation)

Page: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_forkserver_preload

Source: Doc/library/multiprocessing.rst

Current text

Set a list of module names for the forkserver main process to attempt to import so that their already imported state is inherited by forked processes. Any ImportError when doing so is silently ignored. This can be used as a performance enhancement to avoid repeated work in every process.

"their already imported state is inherited" is accurate for each named module, but does not say whether that module's dependencies are inherited too. Where a dependency is imported inside a function body rather than at module level, it is not, and it is imported again in every child.

Demonstration

Two stdlib examples, both reached through ordinary public calls, where the import happens inside the callee:

• datetime.datetime.strptime() imports _strptime on first call. That pulls in 12 modules: _strptime, re (with re._casefix, re._compiler, re._constants, re._parser), locale, _locale, calendar, enum, copyreg and _sre.

• Reading importlib.metadata.Distribution.metadata imports importlib.metadata._adapters and ._text. That pulls in 26 modules, mostly the email package (email.message, email.parser, email.feedparser, email.charset, email.header, email.utils and others), plus base64, quopri, urllib.parse, ipaddress, string, math and datetime.

Neither set is reachable by naming a module in the preload list, because neither import happens at module level.

Measured on 3.14.7, Linux x86-64, fresh interpreter per case, 9 runs each:

	modules added	private memory
import datetime	2	228 kB
first datetime.strptime() call	12	3872 kB
import importlib.metadata	75	6948 kB
first Distribution.metadata read	26	2220 kB

Module counts are exact and reproduced identically on every run. The memory figures are arena-granular, so expect a few hundred kB of variation between environments; the metadata figure also depends on the size of the first distribution's METADATA file (64 KB in this environment). So naming both parent modules in the preload list still leaves 38 modules and roughly 5.9 MiB to be allocated privately in each child. The forkserver process is minimal by design, so these transitive imports are not already satisfied there. In an application running 100 children this is about 600MiB.

Reproducer

"""set_forkserver_preload() does not cover lazily imported dependencies.

$ python3 forkserver_lazy_preload.py           # modules preloaded by name
$ WARM=1 python3 forkserver_lazy_preload.py    # ... plus one call to each
"""

from __future__ import annotations

import datetime
import importlib.metadata
import multiprocessing as mp
import os
import sys

# Imported inside datetime.strptime() and Distribution.metadata respectively,
# so naming their parent modules in the preload list does not pull them in.
LAZY = ("_strptime", "importlib.metadata._adapters")
PAGE_KB = os.sysconf("SC_PAGE_SIZE") // 1024


def touch():
    """Two ordinary calls whose imports happen inside the callee."""
    datetime.datetime.strptime("2026-01-01", "%Y-%m-%d")
    next(iter(importlib.metadata.distributions())).metadata["Name"]


# Runs in the forkserver as well, because "__main__" is in the preload list.
if int(os.environ.get("WARM", "0")) == 1:
    touch()


def rss_kb():
    with open("/proc/self/statm") as statm:
        return int(statm.read().split()[1]) * PAGE_KB


def child(queue):
    inherited = sum(name in sys.modules for name in LAZY)
    before = rss_kb()
    touch()
    queue.put((inherited, rss_kb() - before))


if __name__ == "__main__":
    mp.set_start_method("forkserver")
    mp.set_forkserver_preload(["__main__", "datetime", "importlib.metadata"])

    queue = mp.Queue()
    children = [mp.Process(target=child, args=(queue,)) for _ in range(10)]
    for proc in children:
        proc.start()
    rows = [queue.get() for _ in children]
    for proc in children:
        proc.join()

    print(f"python {sys.version.split()[0]}  WARM={os.environ.get('WARM', '0')}")
    print(f"  lazy deps present in child : {sum(r[0] for r in rows)}/{2 * len(rows)}")
    print(f"  private memory per child   : {sum(r[1] for r in rows) // len(rows)} kB")

datetime and importlib.metadata are named in the preload list explicitly, to show that naming them is not sufficient. WARM=1 additionally calls each function at module level, which — because "__main__" is preloaded — runs in the forkserver process.

❯ WARM=0 python3.14 forkserver_lazy_preload.py
  lazy deps present in child : 0/20
  private memory per child   : 1852 kB


❯ WARM=1 python3.14 forkserver_lazy_preload.py
  lazy deps present in child : 20/20
  private memory per child   : 576 kB

Proposed addition

Appended to the set_forkserver_preload() entry:

 Only the modules named in *module_names* are imported. If one of them imports a further
 module from inside a function body rather than at module level, that further module is
 not imported in the forkserver process, and is imported again in each child. For
 example, listing ``"datetime"`` imports :mod:`datetime`, while :mod:`!_strptime` and
 the :mod:`re`, :mod:`locale` and :mod:`calendar` modules it imports are loaded by the
 first call to :meth:`~datetime.datetime.strptime` in each child process. To inherit
 those as well, call such a function at the module level of a preloaded module, so that
 the import happens in the forkserver process.

Related issues

gh-117378, gh-98552 and gh-141860 concern preload imports that fail silently (ImportError swallowed, sys_path ignored, main_path renamed). This report concerns preload imports that succeed exactly as documented, which no error reporting would surface.

Environment

Python 3.14.7, Linux x86-64.

Linked PRs

• gh-157229
• gh-157237

Modificate la voce set_forkserver_preload() in Doc/library/multiprocessing.rst. Leggete la formulazione attuale e l’aggiunta proposta in questa issue, quindi aggiornate la documentazione per spiegare che gli import nel corpo di una funzione non vengono ereditati e come possono essere precaricati; la documentazione deve riflettere accuratamente l’esempio datetime e il comportamento dimostrato.