pybind / pybind/pybind11

[BUG]: 2 successive py::scoped_interpreter -> double free or corruption (out)

Open
#5,412 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

v2.13

Problem description

I get: double free or corruption (out)

Happens when having 2 successive non-overlapping py::scoped_interpreter (and doing module imports):

{
    py::scoped_interpreter inter;
    py::object imported = py::module::import("mysum"); // mysum.py imports e.g. numpy
}

{
    py::scoped_interpreter inter;                      // again
    py::object imported = py::module::import("mysum"); // crash !   double free or corruption (out)
}
Reproducible example code

Reproducible example: https://github.com/user706/pybind11_test

  • Fails only with numpy and py::array_t and "single" py::scoped_interpreter
    (~> branch main)
  • Does not fail, when having only single py::scoped_interpreter
    (~> branch works_when_having_only_single_scoped_interpreter -> see diff): why?
  • Does not fail, when changing to py::list
    (~> branch works_when_removing_numpy -> see diff): why? (perhaps because of no import)

Do this:

git clone https://github.com/user706/pybind11_test
cd pybind11_test
mkdir build/
cd    build/
cmake ..              # will clone https://github.com/pybind/pybind11.git (stable)
cmake --build . -j    # will also copy mysum.py into the build/ directory
./go                  # run (need a python with numpy)

result:

Sum is: 15
double free or corruption (out)
Aborted (core dumped)

It fails in this line, but only the 2nd time through the loop!!!

But if I simply change the following line
constexpr int NUM_LOOPS = 2;
to this
constexpr int NUM_LOOPS = 1;
then it will not fail.

Why does it fail with NUM_LOOPS = 2?
How can one fix it?

Thanks

Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by cloning the linked reproducer, building it with CMake, and running ./go with NumPy installed. Inspect main.cpp around lines 28 and 32, then compare the main branch with the single-interpreter and no-NumPy branches; done means two successive scoped interpreters complete without double-free corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, numpy, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.