Improve detection of `Py_DECREF` on dictionaries in freelist

Crash report

What happened?

After updating my extension library from Python 3.11 to 3.14, I had a strage segmentation fault raised by my tests. The analysis of this issue took me about a week. The problem here was the introduction of freelists for Python Dictionaries:

When a Python dictionary is freed (refcnt == 0), it get's appended to a list of "free" objects, which may get reused by subsequent PyDict_New() calls. This freelist is implemented as a linked LIFO list. The link between the elements in this list is done by "reusing" the reference counter of the dictionary and storing the address of the next elements in the list in it. This can be problematic because of two reasons:

First of all, if I accidentally Py_DECREF a dictionary too many times, the reference counter does not become negative and hence, this issue is not detected. This is, because the reference counter will container a pointer to the next element as soon as it gets zero. Hence, the next call of Py_DECREF will read the pointer address as integer and assume it is just very large (or in best case negative, in which case it may fail).
This is bad, as it hardens bug detection in debug builds.

But the real problem araises, if the integer representation of the pointer in this dict is not larget than the minimum reference count for immortals: If the integer is larger than this value, the dict is counted as immortal and nothing happens - happy case. If not, Py_DECREF (or Py_INCREF) will modify the reference count and thus, accidentally modifies the pointer! At this point the pointer is off by one and the freelist is broken.

Now, if two new dictionaries are requested, this can cause a segmentation fault on the second dict, because the process might read one byte behind its memory. In best case, this pops up near to the place, where the dict was decref'd, but in worse case, this situation can linger for a long time in the program and pop up in a completely unrelated place. E.g. even when doing d = {} in the Python code.

The following code tries to demonstrate this problem:

#include <Python.h>

void main() {
    PyObject *d1, *d2, *dict;
    Py_Initialize();

    // populate the freelist with two dicts
    d1 = PyDict_New();
    Py_DECREF(d1);

    d2 = PyDict_New();
    Py_DECREF(d2);

    // Now decref a dict too many times
    // For this, first create a new dict. It will take the first entry from the freelist (d2).
    dict = PyDict_New();
    assert(dict == d2);
    assert(Py_REFCNT(dict) == 1);

    // Now if we decref this dict, the reference counter will be overwritten
    // with the next element in the freelist (d1).
    Py_DECREF(dict);
    assert(Py_REFCNT(dict) == (Py_ssize_t)d1);

    // Now comes the bug: 
    // IFF (int)d1 < _Py_IMMORTAL_MINIMUM_REFCNT, the dict is not considered immortal
    // and a Py_DECREF will accidentally modify the address pointer
    Py_DECREF(dict);
    assert(PyUnstable_IsImmortal(dict) || Py_REFCNT(dict) == (Py_ssize_t)d1 - 1);
}

The output of this depends on the address of d1 and whether it's integer representation is larger than _Py_IMMORTAL_MINIMUM_REFCNT or not.

CPython versions tested on:

3.14

Operating systems tested on:

Linux, Windows

Output from running 'python -VV' on the command line:

Python 3.14.1 (main, Dec 2 2025, 19:40:56) [Clang 21.1.4 ]

Inizia riproducendo il comportamento con l’esempio C++ dell’issue su CPython 3.14, quindi traccia la gestione della freelist dei dizionari e il comportamento di Py_DECREF/Py_INCREF nel codice sorgente di CPython. L’issue non indica file o test specifici e non sono specificati criteri di completamento oltre al miglioramento del rilevamento della corruzione.