Unclosed interpreter causes segfault after fork() in child process.

Bug report

Bug description:

With concurrent.interpreters now available in 3.14 it's possible to trigger a probably already known issue with cleaning up interpreters after a fork.

This minimal example creates an interpreter (on the same thread), doesn't close it and then forks. The child process segfaults.

#!/usr/bin/python3.14

import os
from concurrent import interpreters

GLOBAL_INTERP = None

def create_subinter(do_close: bool) -> None:
    global GLOBAL_INTERP
    interp = interpreters.create()
    GLOBAL_INTERP = interp
    interp.exec("""
x = 42 * 23
""")
    if do_close:
        interp.close()


def main():
    global GLOBAL_INTERP

    create_subinter(do_close=False)
    print("Before fork")
    child_pid = os.fork()

    if child_pid == 0:
        print("in child")
    else:
        print("in parent")

if __name__ == "__main__":
    main()

There seems to be a more general issue with the implementation of PyOS_AfterFork_Child() or PyInterpreterState_Clear() specifically:

https://github.com/python/cpython/blob/a15aeec29efa5b3d5d5568278c13bb3fc45f52ef/Python/pystate.c#L992-L994

As the comment points out, the PyInterpreterState_Clear() won't be able to clear any interpreters, as the tstate is set to NULL before calling the function. Which finally results in a Py_DECREF on NULL.

I guess the expected behavior is that manually closing interpreters before a fork is not required and a working clear in the fork. While the combination of subinterpreters and fork() seems a bit obscure, we already ran into this issue in a CI setup.

It also seems that there is a lack of test coverage with the mentioned implementation in pystate.c. If appreciated, I could also follow up with a pull request adding more tests.

Full backtrace:

#0  _Py_DECREF_DecRefTotal () at Objects/object.c:287
#1  0x0000555555816177 in Py_DECREF (filename=filename@entry=0x555555908674 "Python/import.c", lineno=lineno@entry=4099, op=0x7ffff6ea0230) at ./Include/refcount.h:400
#2  0x000055555581d0af in _PyImport_ClearCore (interp=interp@entry=0x7ffff7a59020) at Python/import.c:4099
#3  0x000055555584a008 in PyInterpreterState_Clear (interp=interp@entry=0x7ffff7a59020) at Python/pystate.c:986
#4  0x000055555584a7a3 in _PyInterpreterState_DeleteExceptMain (runtime=runtime@entry=0x555555b809c0 <_PyRuntime>) at Python/pystate.c:1085
#5  0x000055555588f202 in PyOS_AfterFork_Child () at ./Modules/posixmodule.c:741
#6  0x000055555588f49a in os_fork_impl (module=<optimized out>) at ./Modules/posixmodule.c:8159
#7  0x000055555588f516 in os_fork (module=<optimized out>, _unused_ignored=_unused_ignored@entry=0x0) at ./Modules/clinic/posixmodule.c.h:4394
#8  0x00005555556ce81a in cfunction_vectorcall_NOARGS (func=0x7ffff711d3d0, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:508
#9  0x0000555555676df0 in _PyObject_VectorcallTstate (tstate=0x555555bd1938 <_PyRuntime+331640>, callable=callable@entry=0x7ffff711d3d0, args=args@entry=0x7fffffffd628,
    nargsf=9223372036854775808, kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:169
#10 0x0000555555676ecc in PyObject_Vectorcall (callable=callable@entry=0x7ffff711d3d0, args=args@entry=0x7fffffffd628, nargsf=<optimized out>, kwnames=kwnames@entry=0x0)
    at Objects/call.c:327
#11 0x00005555557b4ca6 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7fb2080, throwflag=0) at Python/generated_cases.c.h:1619
#12 0x00005555557d3d57 in _PyEval_EvalFrame (tstate=tstate@entry=0x555555bd1938 <_PyRuntime+331640>, frame=frame@entry=0x7ffff7fb2020, throwflag=throwflag@entry=0)
    at ./Include/internal/pycore_ceval.h:119
#13 0x00005555557d3f28 in _PyEval_Vector (tstate=tstate@entry=0x555555bd1938 <_PyRuntime+331640>, func=func@entry=0x7ffff716ab10, locals=locals@entry=0x7ffff71782f0,
    args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1965
#14 0x00005555557d4024 in PyEval_EvalCode (co=co@entry=0x7ffff7bd23b0, globals=globals@entry=0x7ffff71782f0, locals=locals@entry=0x7ffff71782f0) at Python/ceval.c:857
#15 0x000055555584b13f in run_eval_code_obj (tstate=tstate@entry=0x555555bd1938 <_PyRuntime+331640>, co=co@entry=0x7ffff7bd23b0, globals=globals@entry=0x7ffff71782f0,
    locals=locals@entry=0x7ffff71782f0) at Python/pythonrun.c:1365
#16 0x000055555584b2dc in run_mod (mod=mod@entry=0x555555d7f060, filename=filename@entry=0x7ffff7198400, globals=globals@entry=0x7ffff71782f0, locals=locals@entry=0x7ffff71782f0,
    flags=flags@entry=0x7fffffffda28, arena=arena@entry=0x7ffff71e4a00, interactive_src=0x0, generate_new_source=0) at Python/pythonrun.c:1459
--Type <RET> for more, q to quit, c to continue without paging--                          
#17 0x000055555584bb5c in pyrun_file (fp=fp@entry=0x555555c49a60, filename=filename@entry=0x7ffff7198400, start=start@entry=257, globals=globals@entry=0x7ffff71782f0,
    locals=locals@entry=0x7ffff71782f0, closeit=closeit@entry=1, flags=0x7fffffffda28) at Python/pythonrun.c:1293
#18 0x000055555584d3f0 in _PyRun_SimpleFileObject (fp=fp@entry=0x555555c49a60, filename=filename@entry=0x7ffff7198400, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffda28)
    at Python/pythonrun.c:521
#19 0x000055555584d5e6 in _PyRun_AnyFileObject (fp=fp@entry=0x555555c49a60, filename=filename@entry=0x7ffff7198400, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffda28)
    at Python/pythonrun.c:81
#20 0x0000555555876b67 in pymain_run_file_obj (program_name=program_name@entry=0x7ffff71b2640, filename=filename@entry=0x7ffff7198400, skip_source_first_line=0) at Modules/main.c:410
#21 0x0000555555876c8f in pymain_run_file (config=config@entry=0x555555b9c9f0 <_PyRuntime+114736>) at Modules/main.c:429
#22 0x00005555558777ca in pymain_run_python (exitcode=exitcode@entry=0x7fffffffdba4) at Modules/main.c:694
#23 0x0000555555877a7f in Py_RunMain () at Modules/main.c:775
#24 0x0000555555877af6 in pymain_main (args=args@entry=0x7fffffffdc00) at Modules/main.c:805
#25 0x0000555555877bc4 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:829
#26 0x00005555555de166 in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-145545

從連結的 PR gh-145545 開始，然後在 Python 3.14 上使用 concurrent.interpreters 和 os.fork 重現崩潰。檢查 Python/pystate.c 中的 PyOS_AfterFork_Child() 和 PyInterpreterState_Clear()，以及 Modules/posixmodule.c 中的 fork 路徑。子程序在沒有發生 segfault 的情況下結束，且回歸測試涵蓋了跨 fork() 的未關閉直譯器，即表示完成。