Pybind incorrectly deregisters instances (by type comparison), yields errors when C++ recycles memory
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 18k
- Forks
- 2.3k
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 10
Description
I ran into an interesting (read: Heisen) bug that occurred when I had code that looked like this:
obj = m.UniquePtrHeld(1)
m.unique_ptr_terminal(obj) # Destroys the value held by `obj`
obj = m.UniquePtrHeld(1)
After C++ destroyed the first instance, the second instance was allocated into the same address, when casting the instance back, pybind registered a new instance (in get_internals().registered_instances) when the same ptr-value, but a different instance.
In deregister_instance_impl, it finds all values that match the given ptr, and only checks if the Py_TYPE is the same.
The fix in this case is just to compare instances (if there wasn't a strong reason for getting the type). An example change:
https://github.com/EricCousineau-TRI/pybind11/blob/feature/unique_ptr_arg_pr1-wip/include/pybind11/detail/class.h#L225
Even though I encountered this for my unique_ptr PRs, I believe it could still be an issue when returning bare pointers, say:
// c++
struct MyType { int value; }
...
m.def("do_something", [](MyType *in) {
delete in;
return new MyType{10};
});
# python
obj = m.MyType(10)
obj = do_something(obj);
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in include/pybind11/detail/class.h around line 225, then trace deregister_instance_impl and the registered_instances lookup. Reproduce the pointer-address reuse examples with unique_ptr and bare pointers, and verify that deregistration distinguishes the original instance from a new object at the same address without producing errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100