subtype_dealloc for heap types is a footgun

Bug report

Bug description:

#include <Python.h>

static PyType_Slot base_slots[] = {
    {0, 0}
};

static PyType_Spec base_spec = {
    .name = "Base",
    .basicsize = 0,
    .itemsize = 0,
    .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
    .slots = base_slots,
};

static void derived_dealloc(PyObject *o) {
    // call base class dealloc
    destructor base_slot = PyType_GetSlot(Py_TYPE(o), Py_tp_dealloc);
    base_slot(o);
}

static PyType_Slot derived_slots[] = {
    {Py_tp_dealloc, derived_dealloc},
    {0, 0}
};

static PyType_Spec derived_spec = {
    .name = "Derived",
    .basicsize = 0,
    .itemsize = 0,
    .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
    .slots = derived_slots,
};

static int
deallocdemo_module_exec(PyObject *m)
{
    PyObject *base_type = PyType_FromModuleAndSpec(
        m,
        &base_spec,
        NULL
    );
    if (!base_type) return -1;
    if (PyModule_AddObjectRef(m, "Base", base_type) < 0) {
        Py_DECREF(base_type);
        return -1;
    }

    PyObject *derived_type = PyType_FromModuleAndSpec(
        m,
        &derived_spec,
        base_type
    );
    Py_DECREF(base_type);
    if (!derived_type) return -1;
    int add_object_res = PyModule_AddObjectRef(m, "Derived", derived_type);
    Py_DECREF(derived_type);

    return add_object_res;
}

static PyModuleDef_Slot deallocdemo_module_slots[] = {
    {Py_mod_exec, deallocdemo_module_exec},
    {0, NULL}
};

static struct PyModuleDef deallocdemo_module = {
    .m_base = PyModuleDef_HEAD_INIT,
    .m_name = "deallocdemo",
    .m_size = 0,
    .m_slots = deallocdemo_module_slots,
};

PyMODINIT_FUNC
PyInit_deallocdemo(void)
{
    return PyModuleDef_Init(&deallocdemo_module);
}

Essentially, subtype_dealloc prevents any derived type from implementing its own dealloc function. In the example above, trying to destroy a Derived object will cause an infinite loop because subtype_dealloc will call derived_dealloc (which calls subtype_dealloc).

This is something that works better for static-types, where the default was to inherit the tp_dealloc of the base class. I appreciate heap types are a little more complex in the inheritance they allow though.

I suspect this is now too late to change easily, and I can't personally think of any obvious solutions to it. It's obvious possible to work around (by creating a custom tp_dealloc at all levels) so the issue is more "it's a footgun" than "it's unworkable". But it seems worth raising the issue in case it can be improved somehow.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

The reproducer uses PyType_FromModuleAndSpec, PyType_GetSlot, Py_tp_dealloc, and subtype_dealloc; begin by tracing that deallocation path in CPython's type implementation. Establish and test a safe behavior for heap-type deallocators using the supplied Base/Derived extension as the regression case; the issue does not name a test file or prescribe a solution.