pybind / pybind/pybind11

Consider making the default policy for `const T&` arguments more friendly for non-copyable classes in callbacks

Open
#1,241 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Follow-up from #1240 (#1200).

We disable copying and moving (not that moving is relevant here) for a set of virtual classes, because we would like to have a compile-time guarantee that we are not slicing classes. This generally has no issue in pybind, until callbacks are used, in which case the const T& instance may not be registered, and thus pybind tries to copy it, and then throws a runtime error.

Potential solutions:

  • Specify callback arguments' return-value-policies (per @jagerman's suggestion) when casting a function.
  • Modify the type_caster_base::cast(const T&) to check for automatic and if the class is non-copyable and non-movable, default to reference rather than copy.
    • This seems like it might be backwards-incompatible, but possibly not (as copying would be unsuccessful anyways).

My workaround for the time being will be to wrap any callbacks (Python callbacks cast into std::function<...>) that have const T& references to use const T* instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the related issues #1240 and #1200 first, then inspect type_caster_base::cast(const T&) and the callback-casting path mentioned in the issue. Decide between callback return-value policies and automatic-to-reference behavior for non-copyable types, and add regression coverage showing that a const-reference callback avoids the runtime copy error without breaking existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.