pybind / pybind/pybind11

Enhancement: easy way to expose property with automatic reference counting

Open
#1,096 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Something that I've run into and which has also come up a few times on Gitter: if a class A stores a pointer to an instance of class B (whether raw or smart), it is desirable to be able to expose either the data member or the getter and setter in a way that automatically manages the reference counting so that the Python wrapper for A keeps alive the Python wrapper for B. This is currently tricky to do right, even if one ignores the case of an existing instance of A being returned to Python rather than constructed from Python:

  • keep_alive sort-of works, but it will keep alive every B that is ever set on A, rather than just the latest version.
  • An alias class of A which embeds a py::object to keep B alive works, but IIRC, there are issues with alias classes if the base class is not polymorphic.
  • Embedding the py::object directly into class A works, but only if you have the freedom to modify the class as part of the writing the bindings.
  • You can write a wrapper for the setter than finds the Python objects corresponding to the new and old value, and incs and decs their refcounts directly. That has the advantage of not requiring any extra storage, but will leak the ref on destruction of A unless the destructor behaviour is somehow overridden.

I suggest that some new call policy be added that can do something like the last option but also take care of destruction properly, as well as do something sensible in the case that an existing A is returned (and to be honest, I'm not sure what the most sensible behaviour is).

I suspect this will need to be somewhat intrusive, e.g. adding a structure to the internals to keep track of references (similar to the strong refs used by keep_alive, but indexed by some sort of "slot" label rather than a list).

I wouldn't mind trying to take this on, but I'd first like some guidance on what the interface should look like.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing keep_alive policy, strong-reference handling, and alias-class behavior described in the issue. Clarify the call-policy interface and lifecycle behavior for replacing and destroying referenced objects, including when an existing A is returned to Python; done requires agreement on that design before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.