pybind / pybind/pybind11

[BUG]: Objects created using __new__ are not registered for casting

Open
#6,025 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

3.0.1

Problem description

When a pybind-based object or a subclass of one is created using new, it is not added to the registry of pybind objects for casting. If the object has methods than require casting to return the correct type information for its actual subtype, then this silently breaks those methods.

This is somewhat related to issue #1693, as using the listed workaround is how I found this problem.

Reproducible example code
Suppose we have:

#include <pybind11/pybind11.h>

using namespace py = pybind11;

struct Num {
    double n;
    Num(double n) : n(n) {}
    
    inline Num Mult2() { return Num(2 * n) }
}

PYBIND11_MODULE(this_example, m) {
    py::class_<Num> pyNum(m, "Num");

    pyNum
    .def(py::init([]() { return Num(0); }))
    .def("mult2", [](const Num *self) {
        py::object selfObj = py::cast(self);
        PyTypeObject *const ty = Py_TYPE(selfObj.ptr());
        auto newObj = py::reinterpret_steal<py::object>(ty->tp_new(ty, py::list().ptr(), py::dict().ptr()));
        *newObj.cast<Num*>() = self->Mult2();
        return newObj;
    })
    .def_readwrite("n", &Num::n);
}



from this_example import Num

class MoreNum(Num):
    def mult4(self):
        ret = self.__class__.__new__(self.__class__)
        ret.n = self.n * self.n
        return ret

# Works, but returned object is not registered.
MoreNum().mult2()
MoreNum().mult4()
MoreNum().mult2().mult4()

# mult4 is not implemented on Num
MoreNum().mult2().mult2().mult4()
Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by building the reproducible C++ pybind11 example and running the shown Python calls involving Num, MoreNum, new, mult2, and mult4. Trace how py::class_ objects created through new enter the pybind11 object registry; done means these objects are registered and chained calls preserve the actual subtype behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.