pybind / pybind/pybind11

Pybind incorrectly deregisters instances (by type comparison), yields errors when C++ recycles memory

Open
#1,238 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I ran into an interesting (read: Heisen) bug that occurred when I had code that looked like this:

obj = m.UniquePtrHeld(1)
m.unique_ptr_terminal(obj)  # Destroys the value held by `obj`
obj = m.UniquePtrHeld(1)

After C++ destroyed the first instance, the second instance was allocated into the same address, when casting the instance back, pybind registered a new instance (in get_internals().registered_instances) when the same ptr-value, but a different instance.
In deregister_instance_impl, it finds all values that match the given ptr, and only checks if the Py_TYPE is the same.

The fix in this case is just to compare instances (if there wasn't a strong reason for getting the type). An example change:
https://github.com/EricCousineau-TRI/pybind11/blob/feature/unique_ptr_arg_pr1-wip/include/pybind11/detail/class.h#L225

Even though I encountered this for my unique_ptr PRs, I believe it could still be an issue when returning bare pointers, say:

// c++
struct MyType { int value; }
...
m.def("do_something", [](MyType *in) {
  delete in;
  return new MyType{10};
});

# python
obj = m.MyType(10)
obj = do_something(obj);

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in include/pybind11/detail/class.h around line 225, then trace deregister_instance_impl and the registered_instances lookup. Reproduce the pointer-address reuse examples with unique_ptr and bare pointers, and verify that deregistration distinguishes the original instance from a new object at the same address without producing errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.