tracemalloc: Heap-use-after-free in _Py_IsImmortal when destroying subinterpreters while tracing

Crash report

What happened?

from concurrent import interpreters
import tracemalloc

# Start tracking memory allocations.
# Tracemalloc captures filenames of code objects loaded during execution.
tracemalloc.start()

# Create a subinterpreter.
# During initialization (specifically _Py_Get_Getpath_CodeObject), 
# the subinterpreter allocates and interns strings (filenames).
# Tracemalloc records pointers to these strings.
interp_id = interpreters.create()

# Ensure the interpreter is fully initialized/used.
interpreters.run_string(interp_id, "pass")

# Destroy the subinterpreter.
# This clears the subinterpreter's interned dictionary, freeing the strings 
# that tracemalloc is still tracking.
interpreters.destroy(interp_id)

# When the script exits, _Py_Finalize calls tracemalloc_deinit.
# Tracemalloc iterates its traces and tries to access the filename strings 

<sys>:0: RuntimeWarning: remaining subinterpreters; close them with Interpreter.close()
=================================================================
==2128302==ERROR: AddressSanitizer: heap-use-after-free on address 0x6f04cd8396e0 at pc 0x569c61046bb6 bp 0x7ffd6872cba0 sp 0x7ffd6872cb98
READ of size 4 at 0x6f04cd8396e0 thread T0
    #0 0x569c61046bb5 in _Py_IsImmortal /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/refcount.h:129:12
    #1 0x569c61046bb5 in Py_DECREF /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/refcount.h:414:9
    #2 0x569c61046bb5 in tracemalloc_clear_filename /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:706:5
    #3 0x569c60eba2e0 in _Py_hashtable_destroy_entry /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/hashtable.c:382:9
    #4 0x569c60eba2e0 in _Py_hashtable_clear /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/hashtable.c:398:13
    #5 0x569c61047edf in tracemalloc_clear_traces_unlocked /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:721:5
    #6 0x569c61047edf in _PyTraceMalloc_Stop /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:877:5
    #7 0x569c61048f57 in tracemalloc_deinit /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:783:5
    #8 0x569c61048f57 in _PyTraceMalloc_Fini /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:1261:5
    #9 0x569c60fe7fd6 in _Py_Finalize /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:2301:5
    #10 0x569c6106a07a in Py_RunMain /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:774:9
    #11 0x569c6106b943 in pymain_main /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:802:12
    #12 0x569c6106baa2 in Py_BytesMain /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:826:12
    #13 0x72a4ce73bd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x72a4ce73be3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x569c60884e94 in _start (/home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/python+0x1fce94) (BuildId: 0e20bf7695762228d93d1548a3b79cafed8ba475)

0x6f04cd8396e0 is located 0 bytes inside of 52-byte region [0x6f04cd8396e0,0x6f04cd839714)
freed by thread T0 here:
    #0 0x569c609296e6 in free (/home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/python+0x2a16e6) (BuildId: 0e20bf7695762228d93d1548a3b79cafed8ba475)
    #1 0x569c6104789a in tracemalloc_free /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:643:5
    #2 0x569c60c867e1 in unicode_dealloc /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:1704:5
    #3 0x569c60b77d8d in _Py_Dealloc /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/object.c:3207:5
    #4 0x569c60b3e57a in Py_DECREF /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/refcount.h:420:9
    #5 0x569c60b3e57a in Py_XDECREF /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/refcount.h:513:9
    #6 0x569c60b3e57a in dictkeys_decref /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/dictobject.c:461:17
    #7 0x569c60c8c6ce in clear_interned_dict /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:317:13
    #8 0x569c60c8c6ce in _PyUnicode_ClearInterned /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:14436:5
    #9 0x569c60febf65 in finalize_interp_types /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:1907:5
    #10 0x569c60febf65 in finalize_interp_clear /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:1951:5
    #11 0x569c60fe8fc6 in Py_EndInterpreter /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:2611:5
    #12 0x569c60fe9701 in finalize_subinterpreters /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:2675:9
    #13 0x569c60fe9701 in make_pre_finalization_calls /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:2133:13
    #14 0x569c60fe7e62 in _Py_Finalize /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/pylifecycle.c:2181:5
    #15 0x569c6106a07a in Py_RunMain /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:774:9
    #16 0x569c6106b943 in pymain_main /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:802:12
    #17 0x569c6106baa2 in Py_BytesMain /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Modules/main.c:826:12
    #18 0x72a4ce73bd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x569c60929984 in malloc (/home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/python+0x2a1984) (BuildId: 0e20bf7695762228d93d1548a3b79cafed8ba475)
    #1 0x569c6104afa6 in tracemalloc_alloc /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/tracemalloc.c:518:15
    #2 0x569c60c39be3 in PyUnicode_New /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:1320:24
    #3 0x569c60c40520 in _PyUnicode_FromUCS1 /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:2164:11
    #4 0x569c60c40520 in PyUnicode_FromKindAndData /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/unicodeobject.c:2280:16
    #5 0x569c60fa147c in r_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1309:17
    #6 0x569c60fa0f8a in r_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1570:24
    #7 0x569c60fa0e06 in r_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1382:18
    #8 0x569c60fa0f2f in r_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1558:22
    #9 0x569c60f9a792 in read_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1718:9
    #10 0x569c60f9a584 in PyMarshal_ReadObjectFromString /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/marshal.c:1837:14
    #11 0x569c60ec0d18 in unmarshal_frozen_code /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:3130:20
    #12 0x569c60ec87af in _imp_get_frozen_object_impl /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:4614:25
    #13 0x569c60ec87af in _imp_get_frozen_object /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/clinic/import.c.h:285:20
    #14 0x569c60de8f25 in _PyEval_EvalFrameDefault /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/generated_cases.c.h:2582:38
    #15 0x569c60dd5514 in _PyEval_EvalFrame /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_ceval.h:118:16
    #16 0x569c60dd5514 in _PyEval_Vector /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:2094:12
    #17 0x569c60a78bfb in _PyObject_VectorcallTstate /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_call.h:136:11
    #18 0x569c60a78bfb in object_vacall /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:823:14
    #19 0x569c60a78706 in PyObject_CallMethodObjArgs /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:890:24
    #20 0x569c60ec30ef in import_find_and_load /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:3807:11
    #21 0x569c60ec30ef in PyImport_ImportModuleLevelObject /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:3888:15
    #22 0x569c60e126de in _PyEval_ImportName /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:2912:16
    #23 0x569c60ddb155 in _PyEval_EvalFrameDefault /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/generated_cases.c.h:6373:31
    #24 0x569c60dd4afd in _PyEval_EvalFrame /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_ceval.h:118:16
    #25 0x569c60dd4afd in _PyEval_Vector /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:2094:12
    #26 0x569c60dd4afd in PyEval_EvalCode /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:673:21
    #27 0x569c60dcaba4 in builtin_exec_impl /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/bltinmodule.c:1193:17
    #28 0x569c60dcaba4 in builtin_exec /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/clinic/bltinmodule.c.h:579:20
    #29 0x569c60dd7482 in _Py_BuiltinCallFastWithKeywords_StackRefSteal /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:846:11
    #30 0x569c60dde10a in _PyEval_EvalFrameDefault /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/generated_cases.c.h:2391:35
    #31 0x569c60dd5514 in _PyEval_EvalFrame /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_ceval.h:118:16
    #32 0x569c60dd5514 in _PyEval_Vector /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/ceval.c:2094:12
    #33 0x569c60a78bfb in _PyObject_VectorcallTstate /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_call.h:136:11
    #34 0x569c60a78bfb in object_vacall /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:823:14
    #35 0x569c60a78706 in PyObject_CallMethodObjArgs /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:890:24
    #36 0x569c60ec30ef in import_find_and_load /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:3807:11
    #37 0x569c60ec30ef in PyImport_ImportModuleLevelObject /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/import.c:3888:15
    #38 0x569c60dc806e in builtin___import___impl /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/bltinmodule.c:285:12
    #39 0x569c60dc806e in builtin___import__ /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Python/clinic/bltinmodule.c.h:110:20
    #40 0x569c60a76054 in _PyObject_VectorcallTstate /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/internal/pycore_call.h:136:11
    #41 0x569c60a76054 in _PyObject_CallFunctionVa /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:552:18
    #42 0x569c60a75d74 in PyObject_CallFunction /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Objects/call.c:574:14

SUMMARY: AddressSanitizer: heap-use-after-free /home/fuzz/WorkSpace/FusionFuzzLoop/projects/cpython/cpython/build/../Include/refcount.h:129:12 in _Py_IsImmortal
Shadow bytes around the buggy address:
  0x6f04cd839400: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fa
  0x6f04cd839480: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
  0x6f04cd839500: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x6f04cd839580: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fa
  0x6f04cd839600: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
=>0x6f04cd839680: fd fd fd fd fd fd fd fa fa fa fa fa[fd]fd fd fd
  0x6f04cd839700: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fa
  0x6f04cd839780: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x6f04cd839800: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
  0x6f04cd839880: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fa
  0x6f04cd839900: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2128302==ABORTING

related: #134604

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

Commencez par Python/tracemalloc.c, en particulier tracemalloc_clear_filename et tracemalloc_deinit, puis examinez l’ordre du nettoyage dans Objects/unicodeobject.c et Python/pylifecycle.c. Exécutez sous AddressSanitizer le reproducer fourni pour le subinterpreter et tracemalloc. C’est terminé lorsque l’arrêt de l’interpréteur n’accède plus aux chaînes de caractères de noms de fichiers libérées et que le reproducer se termine sans heap-use-after-free.