pybind / pybind/pybind11

[BUG] recursive cast in type_caster of non-class type causes a segfault.

Open
#3,064 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I have discussed this issue on gitter and it appears it might be a bug.

I've created a public github repos and a branch called feature/pybind11_cast_bugthat recreates the problem.
https://github.com/awr-paul/pybind11_helloworld/tree/feature/pybind11_cast_bug

Essentially if you use a non-class type with the pybind11::type_caster<> cast can get called recursively and segfault (at least when numpy is imported).

The problem, manifests itself here:
https://github.com/awr-paul/pybind11_helloworld/blob/feature/pybind11_cast_bug/src/pybind11_helloworld/submodule/main.cpp#L71-72

and can be recreated like this:

>>> from pybind11_helloworld.submodule import sometime
>>> sometime()
Segmentation fault (core dumped)

However, if a class type is used instead of a primitive then the code works fine (as demonstrated on the main branch).
https://github.com/awr-paul/pybind11_helloworld/blob/main/src/pybind11_helloworld/submodule/main.cpp#L75

If the pybind11::type_caster<> does not support non-class types then this should be documented and a helpful compile error message could be added to the type_caster rather than allowing it to compile and segfault:

static_assert(std::is_class<T>::value, "pybind11::type_caster only supports class types");

However, I suspect the issue is another bug, possibly with how pybind11 interacts with the numpy ABI?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the crash with the linked feature/pybind11_cast_bug branch, importing numpy and calling sometime(), then inspect src/pybind11_helloworld/submodule/main.cpp around lines 71–72 and compare the class case around line 75. Determine whether non-class pybind11::type_caster<> inputs are supported; done means the example no longer segfaults and the supported behavior is documented or diagnosed at compile time.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, numpy, python
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.