pybind / pybind/pybind11

Automatic custom holder type encapsulation

Open
#1,219 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I apologise for the duplication of this question, I had already inquired about this on Gitter but with no success.

I've recently been experimenting with pybind with the intent to use it in wrapping a fairly substantial existing codebase.
Certain methods, bound in python, will return objects that can potentially be concurrently destroyed in c++ during script execution. Smart pointers are not used in the c++ codebase, the objects are destroyed using delete on raw pointers.
I was hoping to find a mechanic that would allow me to automatically encapsulate these objects, in python, in a container that would be notified upon object destruction and set its internal reference to None. I want to minimise as much as possible the need to modify existing methods to accommodate the binding process.
The custom holder type described here appeared to fill this criteria but I can not get this to work as I desire. I think I haven't fully understood their purpose and/or underlying mechanics, so I'd be happy with any help regarding custom holder types or any other feature that might achieve the same result.

Here is my c++ code + binding declaration for testing the idea

template< typename T >
class SmartPointer{
public:
    SmartPointer(T * pointer)
        : m_Pointer(pointer) {
        pybind11::print("c++ - ", (size_t)(void*)this, "SmartPointer::SmartPointer(", (size_t)m_Pointer, ")");
        m_Pointer->DeleteEvent = [&] () {
            pybind11::print("c++ - ", (size_t)(void*)this, "Foo::DeleteEvent");
            m_Pointer = nullptr;
        };
    }

    ~SmartPointer(void) {
        pybind11::print("c++ - ", (size_t)(void*)this, "SmartPointer::~SmartPointer() -", (size_t)m_Pointer);
    }

    T * get(void) const {
        pybind11::print("c++ - ", (size_t)(void*)this, "SmartPointer::get() const ->", (size_t)m_Pointer);
        return m_Pointer;
    }

private:
    T * m_Pointer;
};

PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPointer< T >);

class Foo {
public:

    Foo(const std::string & name)
        : m_Name(name) {
        pybind11::print("c++ - ", (size_t)this, " Foo::Foo(", name, ")");
    }

    ~Foo(void) {
        DeleteEvent();
        pybind11::print("c++ - ", (size_t)this, " Foo::~Foo() - ", m_Name);
    }

    const std::string & GetName(void) const {
        pybind11::print("c++ - ", (size_t)this, " Foo::GetName() - ", m_Name);
        return m_Name;
    }

    void SetName(const std::string & name) {
        m_Name = name;
        pybind11::print("c++ - ", (size_t)this, " Foo::SetName(", name, ")");
    }

    std::function< void (void) > DeleteEvent; // used to notify its custom smart pointer that this Foo instance has been destroyed

private:
    std::string m_Name;
};

void Destroy(Foo * foo) {
    pybind11::print("c++ - Destroy(", (size_t)foo, ")");
    delete foo;
}


PYBIND11_MODULE(test_pybind, module) {
    module.doc() = "pybind11 example plugin";

    pybind11::class_< Foo, SmartPointer< Foo > >(module, "Foo")
        .def(pybind11::init< const std::string & >())
        .def("GetName", &Foo::GetName)
        .def("SetName", &Foo::SetName);

    module.def("Destroy", &Destroy, "Destroys a Foo instance");
}

In the following python test code, I create a Foo instance and then destroy it, but python's reference still points to the instance, access to this reference does not use the get method of the custom holder type "SmartPointer".

from test_pybind import *
f = Foo("moo")
f.GetName()
Destroy(f)
f.GetName() # reference should be none but is dangling instead, get method not used

I have added a test project to illustrate this issue.
TestPyBind11.zip

Any help or clarification would be greatly appreciated, many thanks in advance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the dangling-reference behavior from the attached TestPyBind11.zip and reviewing the custom holder type documentation linked in the issue. Trace the shown SmartPointer, Foo, Destroy, and PYBIND11_DECLARE_HOLDER_TYPE declarations to determine whether the requested destruction notification is supported; done means a clearly scoped resolution or documented limitation with a regression test if a change is appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.