pybind / pybind/pybind11

[BUG]: tp_traverse occasionally executed before C++ ctor

Open
#4,869 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

2.11.1

Problem description

Enabling Py_TPFLAGS_HAVE_GC in custom_type_setup will occasionally call type->tp_traverse before the C++ ctor. Depending on what the C++ object contains, this can be anything from harmless (if the allocation is zero filled and traversing it just visits some nulls), to segfault (if all-zeroes is not a legal state for the object, for example trying to iterate a libstdc++ std::unordered_map), or worse (if the allocation is not zero filled, the worst case is a security hole).

I don't know if this is a bug in pybind11, Python itself (calling _PyObject_GC_TRACK in PyType_GenericAlloc in Objects/typeobject.c seems kinda suspicious to me), or in my code, but if it's the latter, the same bug also exists at https://pybind11.readthedocs.io/en/stable/advanced/classes.html#custom-type-setup .

Reproducible example code
#include <pybind11/pybind11.h>
#include <unordered_set>

namespace py = pybind11;

class funny_class;
std::unordered_set<funny_class*> items_that_exist;
class funny_class {
public:
    funny_class() { items_that_exist.insert(this); }
    ~funny_class() { items_that_exist.erase(this); }
};

PYBIND11_MODULE(example, m) {
    py::class_<funny_class> elem_container(m, "fun",
        py::custom_type_setup([](PyHeapTypeObject* heap_type) {
            PyTypeObject* type = &heap_type->ht_type;
            type->tp_flags |= Py_TPFLAGS_HAVE_GC;
            type->tp_traverse = [](PyObject* self_base, visitproc visit, void* arg) {
                auto& self = py::cast<funny_class&>(py::handle(self_base));
                if (!items_that_exist.count(&self))
                    puts("ERROR: Item was traversed without being constructed.");
                return 0;
            };
            type->tp_clear = [](PyObject* self_base) {
                auto& self = py::cast<funny_class&>(py::handle(self_base));
                if (!items_that_exist.count(&self))
                    puts("ERROR: Item was erased without being constructed.");
                return 0;
            };
        }));
    elem_container.def(py::init<>());
}
import example

a = [(example.fun(), example.fun())[1] for _ in range(10000)]
Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the custom_type_setup example in the issue and reproduce it using the provided C++ module and Python loop. Compare pybind11's type setup with Python's PyType_GenericAlloc and Objects/typeobject.c, then establish whether traversal before construction is caused by pybind11 or Python. Done means the reproduction no longer traverses an unconstructed object, with a regression test or documented upstream limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.