pybind / pybind/pybind11

[BUG]: Custom SharedPtr<T> usually should not be required to have a copy constructible T.

Open
#5,417 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

https://github.com/pybind/pybind11/archive/refs/tags/v2.13.5.tar.gz

Problem description

I have a custom SharedPtr that is basically an std::shared_ptr with some extra functionalities. It actually derives privately from std::shared_ptr.

My custom SharedPtr<T> is declared as a holder type.

PYBIND11_DECLARE_HOLDER_TYPE(T, SharedPtr<T>);

However, when T is not copy-constructible, I get compilation errors when using it: (error happens in "populate" binding)

void init_scene(py::module_& m)
{
  py::class_<SceneRoot, SharedPtr<SceneRoot>>(
      m, "Scene",
      "A scene graph with a message queue that keeps it updated.")
      .def(py::init(&new_scene),
           "Creates an empty scene and associates it to an OGRE SceneManager.")
      .def("populate",
           [](SharedPtr<SceneRoot> self, std::shared_ptr<DocumentTree> doc)
           {self->populate(std::move(self), std::move(doc));},
           "document"_a,
           "Populates the scene with the contents of 'document'.")
      .def("__repr__",
           [](const SceneRoot& s){ return "<SCENE... (put info here)>"; });
}

This happens because since SceneRoot is not copy-constructible (we do not want to inadvertently copy this huge structure!), type_caster<SharedPtr<T>> inherits from move_only_holder_caster. See https://github.com/pybind/pybind11/blob/f7e14e985be167ca158fd3ee2fe5d8a4f175fa87/include/pybind11/cast.h#L862-L864

I do not really know much about use cases, but I do believe this is not correct. In pybind11 we have a custom is_copy_constructible<> trait because pre C++17 stl libraries sometimes report non-copy constructible as copy constructible but later give a compilation error. See commit https://github.com/pybind/pybind11/commit/793726014d4ea879950f5a7c8f7c6684d51e544b. But in case of custom holders, using pybind's own is_copy_constructible is not correct IMO. I have discussed this (https://github.com/pybind/pybind11/discussions/5142) with myself and we both have agreed. :-)

I have worked around the issue with this:

/*
 * A SharedPtr<T> is copy constructible even if T is not!
 * So, PYBIND11_DECLARE_HOLDER_TYPE does not work. :-(
 */
//PYBIND11_DECLARE_HOLDER_TYPE(T, SharedPtr<T>);
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
namespace detail {
  template <typename T>
  class type_caster<SharedPtr<T>>
      : public copyable_holder_caster<T, SharedPtr<T>> {};
  template <typename T>
  struct is_holder_type<T, SharedPtr<T>> : std::true_type {};
}
PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)

Notice that I also had to declare is_holder_type<T, SharedPtr<T>>.

If I am not missing anything, type_caster_holder should use std::is_copy_constructible instead of the recursive pybind11 version.

This would enable std::shared_ptr to not be treated as a special case here
https://github.com/pybind/pybind11/blob/f7e14e985be167ca158fd3ee2fe5d8a4f175fa87/include/pybind11/cast.h#L879
and here
https://github.com/pybind/pybind11/blob/f7e14e985be167ca158fd3ee2fe5d8a4f175fa87/include/pybind11/cast.h#L840

Side Note

I do not understand how std::shared_ptr deals with the fact that is_holder_type<> is IMO supposed to fail (but of course it doesn't!) when T is not copy-constructible:
https://github.com/pybind/pybind11/blob/f7e14e985be167ca158fd3ee2fe5d8a4f175fa87/include/pybind11/cast.h#L886-L887

Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in include/pybind11/cast.h at type_caster_holder, move_only_holder_caster, copyable_holder_caster, and the std::shared_ptr special cases referenced in the issue. Reproduce the failure with a custom SharedPtr whose T is non-copy-constructible, then inspect the holder traits and caster selection. Done means custom holders work without the issue's specialization workaround while existing std::shared_ptr behavior remains valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.