pybind / pybind/pybind11

[BUG] calling multiple pybind11 scoped env which calls cppyy

Open
#5,414 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

2.13.6

Problem description

i have embedded python with my QT (c++) solution. i call a c++ function which uses pybind11 to get me into python, which in turn sets up a cppyy encapsualtion of a simpe c++ function. it works fine, but call it again it crashes. below is the crash report

"
Hello, World!
['/home/simon/Desktop/Code/VirtEnv/lib/python3.12/site-packages/', '/home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/', '', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/usr/local/lib/python3.12/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.12/dist-packages']
here
there
done
free(): invalid size
*** Break *** abort
#0 0x00007f9cb8710893 in __GI___wait4 (pid=4609, stat_loc=stat_loc
entry=0x7fff23a04f68, options=options
entry=0, usage=usage
entry=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007f9cb87109e7 in __GI___waitpid (pid=, stat_loc=stat_loc
entry=0x7fff23a04f68, options=options
entry=0) at ./posix/waitpid.c:38
#2 0x00007f9cb866891b in do_system (line=) at ../sysdeps/posix/system.c:172
#3 0x00007f9cb5ba1c15 in CppyyLegacy::TUnixSystem::StackTrace() () from /home/simon/Desktop/Code/VirtEnv/lib/python3.12/site-packages/cppyy_backend/lib/libCoreLegacy.so
#4 0x00007f9cb5cd94bf in (anonymous namespace)::do_trace (sig=5) at src/clingwrapper.cxx:249
#5 (anonymous namespace)::TExceptionHandlerImp::HandleException (this=, sig=5) at src/clingwrapper.cxx:262
.....
#42 0x00007f9cb8ef0ae6 in ?? () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#43 0x00007f9cb8ef0bf9 in ?? () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#44 0x00007f9cb8ef41bf in PyRun_StringFlags () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#45 0x000055ea7e523055 in pybind11::eval<(pybind11::eval_mode)2> (expr=..., global=..., local=...) at /usr/include/pybind11/eval.h:74
#46 0x000055ea7e4fcb2b in pybind11::exec (expr=..., global=..., local=...) at /usr/include/pybind11/eval.h:89
#47 0x000055ea7e4fb462 in Test002 () at /home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/main.cpp:147
#48 0x000055ea7e4fb1bb in main (argc=1, argv=0x7fff23a09058) at /home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/main.cpp:32
*** Break *** abort
#0 0x00007f9cb8710893 in __GI___wait4 (pid=4848, stat_loc=stat_loc
entry=0x7fff23a04f68, options=options
entry=0, usage=usage
entry=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
#1 0x00007f9cb87109e7 in __GI___waitpid (pid=, stat_loc=stat_loc
entry=0x7fff23a04f68, options=options
entry=0) at ./posix/waitpid.c:38
#2 0x00007f9cb866891b in do_system (line=) at ../sysdeps/posix/system.c:172
#3 0x00007f9cb5ba1c15 in CppyyLegacy::TUnixSystem::StackTrace() () from /home/simon/Desktop/Code/VirtEnv/lib/python3.12/site-packages/cppyy_backend/lib/libCoreLegacy.so
#4 0x00007f9cb5cd93cf in (anonymous namespace)::do_trace (sig=5) at src/clingwrapper.cxx:249
#5 (anonymous namespace)::TExceptionHandlerImp::HandleException (this=, sig=5) at src/clingwrapper.cxx:268
....
#41 0x00007f9cb8e97507 in PyEval_EvalCode () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#42 0x00007f9cb8ef0ae6 in ?? () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#43 0x00007f9cb8ef0bf9 in ?? () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#44 0x00007f9cb8ef41bf in PyRun_StringFlags () from /lib/x86_64-linux-gnu/libpython3.12.so.1.0
#45 0x000055ea7e523055 in pybind11::eval<(pybind11::eval_mode)2> (expr=..., global=..., local=...) at /usr/include/pybind11/eval.h:74
#46 0x000055ea7e4fcb2b in pybind11::exec (expr=..., global=..., local=...) at /usr/include/pybind11/eval.h:89
#47 0x000055ea7e4fb462 in Test002 () at /home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/main.cpp:147
#48 0x000055ea7e4fb1bb in main (argc=1, argv=0x7fff23a09058) at /home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/main.cpp:32
13:33:11: /home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/build/Desktop_Qt_6_7_2-Debug/Test_pybind11_cppyy_001 exited with code 133
"

when run from with a debugger:
"
Hello, World!
['/home/simon/Desktop/Code/VirtEnv/lib/python3.12/site-packages/', '/home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/', '', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/usr/local/lib/python3.12/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.12/dist-packages']
here
there
done
free(): invalid size
44 ./nptl/pthread_kill.c: No such file or directory
81 ../sysdeps/unix/sysv/linux/internal-signals.h: No such file or directory
50 ./nptl/pthread_kill.c: No such file or directory
89 ../sysdeps/unix/sysv/linux/internal-signals.h: No such file or directory
69 ./nptl/pthread_kill.c: No such file or directory
"

Reproducible example code
as called from my QT solution:
"
//
void Test002()
{
//
py::scoped_interpreter guard{};
//
QString qPythonDir("/home/simon/Code/QT/Test_pybind11/Test_pybind11_cppyy_001/");
QString qPythonCommand("");
//
auto locals = py::dict();
auto globals = py::dict();
//
qPythonCommand += "import sys\n";
qPythonCommand += "sys.path.insert(0, '";
qPythonCommand += qPythonDir.toStdString().c_str();
qPythonCommand += "')\n";
qPythonCommand += "sys.path.insert(0, '/home/simon/Desktop/Code/VirtEnv/lib/python3.12/site-packages/')\n";
qPythonCommand += "print(sys.path)\n";
py::exec(qPythonCommand.toStdString().c_str(), py::globals(), locals);
//
qPythonCommand =
R"PY(
import cppyy

cppyy.cppdef("""
#include
void hello()
{
std::cout << "Hello, World!" << std::endl;
}
""")

def runCPP():
global cppyy
print("here")
try:
cppyy.gbl.hello()
print("there")
except Exception as e:
print("Exception: ", e)
print("done")
)PY";

//
py::exec(qPythonCommand.toStdString().c_str(), py::globals(), locals);
//
py::object func = locals["runCPP"];
func();
//
}
"

similar idea to above, but this time no pybind11 and directly from with python. works multiple times.
"
import cppyy

cppyy.cppdef(r"""
#include
void hello() {
std::cout << "Hello, World!" << std::endl;
}
""")

def runCPP():
print("here")
try:
cppyy.gbl.hello()
print("there")
except Exception as e:
print("Exception: ", e)
print("done")
"
Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the second invocation of Test002 in main.cpp, especially around lines 32 and 147, using the embedded Python and cppyy setup shown. Read pybind11::exec and pybind11::eval in eval.h alongside the crash trace. Done means the same operation can be called multiple times without the reported invalid-size abort, with a regression check for repeated calls.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.