`dict` item-iterator: `_PyObject_GC_UNTRACK` on never-tracked iterator under OOM (`dictobject.c`)
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 77.2k
- Fork
- 35.9k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
Creating a dict item-iterator (iter(d.items()) or reversed(d.items())) while
allocations are failing aborts a debug build with:
Objects/dictobject.c:5532: _PyObject_GC_UNTRACK:
Assertion "_PyObject_GC_IS_TRACKED(op)" failed:
object not tracked by the garbage collector
object type name: dict_itemiterator
Fatal Python error: _PyObject_AssertFailed
dictiter_new() allocates the iterator with PyObject_GC_New() but only calls
_PyObject_GC_TRACK(di) at the very end. For item iterators it first allocates
di->di_result via _PyTuple_FromPairSteal(). If that allocation fails (OOM), the
error path runs Py_DECREF(di) on the still-untracked iterator, and
dictiter_dealloc() unconditionally calls _PyObject_GC_UNTRACK(di).
- Debug build: the
_PyObject_GC_IS_TRACKEDassertion fires →Py_FatalError/ abort. - Release build (NDEBUG): the assert is compiled out and untracking a never-tracked
object corrupts the GC list, leading to a later segfault in_Py_Dealloc.
Only item iterators are affected (dict_itemiterator / dict_reverseitemiterator) —
they are the only dict iterators whose di_result requires an allocation between
PyObject_GC_New and _PyObject_GC_TRACK.
This was found via allocation-failure fuzzing as OOM-0006 in the umbrella issue
#151763. Filing a focused issue per that issue's process.
Reproducer
Minimal, stdlib-only, deterministic (requires a --with-pydebug interpreter so
_testcapi.set_nomemory is available):
import faulthandler, _strptime
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(60):
set_nomemory(start)
try:
_strptime._strptime("", "")
except BaseException:
pass
print("done, no crash")
_strptime._strptime("", "") performs an iter(d.items()) once the size-2 tuple
freelist is drained, so _PyTuple_FromPairSteal() reaches the failing allocator and
drives the dictiter_new() OOM error path. (Reducer by @devdanzin; gist:
https://gist.github.com/devdanzin/c809eb4072c0c787c0c890f54ba1c843)
Backtrace (abbreviated)
_PyObject_AssertFailed Objects/object.c
_PyObject_GC_UNTRACK Include/internal/pycore_gc.h:254
dictiter_dealloc (self=dict_itemiterator) Objects/dictobject.c:5532
_Py_Dealloc Objects/object.c
Py_DECREF Include/refcount.h
dictiter_new (itertype=PyDictIterItem_Type) Objects/dictobject.c <- Py_DECREF(di) after di_result alloc fails
PyObject_GetIter Objects/abstract.c
Suggested fix
Either guard the untracked against the never-tracked error path:
if (_PyObject_GC_IS_TRACKED(di)) {
_PyObject_GC_UNTRACK(di);
}
or move _PyObject_GC_TRACK(di) ahead of the fallible di_result allocation (after
initializing di->di_result = NULL so traverse/dealloc stay safe). A PR with the first
approach and a regression test is ready.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
No response
Linked PRs
- gh-152110
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia in Objects/dictobject.c, in dictiter_new() e dictiter_dealloc(), quindi esegui il reproducer fornito _strptime e _testcapi.set_nomemory su una build --with-pydebug. Controlla la PR collegata per il lavoro già in corso. L’attività è completata quando il percorso OOM dell’iteratore degli elementi non causa più un abort né corrompe il GC e un test di regressione copre il fallimento.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- c, python
- Ambito
- backend
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 25/100