pybind / pybind/pybind11

Error in Pybinding resulting a segmentation fault

Open
#2,069 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Issue description

I am trying to build an application of C++ numerical solver OpenFOAM with pybind. I am able to successfully embed the python interpreter in the code and was able to manually build pybind with the C++ application. The build application successfully runs for a single iteration of the solver and produce desired output but as the C++ numerical solver progresses to iteration 2 the pybind results into following;

smoothSolver:  Solving for Ux, Initial residual = 0.012672, Final residual = 1.79478e-06, No Iterations 3
smoothSolver:  Solving for Uy, Initial residual = 0.0617171, Final residual = 9.04058e-06, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.0604563, Final residual = 9.35365e-06, No Iterations 3
Pressure gradient source: uncorrected Ubar = 0.13349, pressure gradient = 4.93435e-05
GAMG:  Solving for p, Initial residual = 0.522434, Final residual = 0.0436122, No Iterations 1
time step continuity errors : sum local = 8.58809e-05, global = -2.7721e-20, cumulative = -2.7721e-20
Pressure gradient source: uncorrected Ubar = 0.13349, pressure gradient = 4.89642e-05
GAMG:  Solving for p, Initial residual = 0.0367536, Final residual = 3.21026e-07, No Iterations 6
time step continuity errors : sum local = 1.05564e-09, global = -8.61765e-20, cumulative = -1.13897e-19
Pressure gradient source: uncorrected Ubar = 0.13349, pressure gradient = 4.88702e-05
Hello, World Python Pybyind..!
['/home/hsharma/WORK/Project_OFTFCoupling/OF_Pybind11/Dynamic_Smagorinsky/Py_Modules', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python37.zip', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7/lib-dynload', '/home/hsharma/.local/lib/python3.7/site-packages', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7/site-packages', '.']
Python Result :3
Python Result :"1.15.0-rc1"
ExecutionTime = 3.41 s  ClockTime = 2 s

SmoothSolver:  Solving for Ux, Initial residual = 0.0120503, Final residual = 5.86065e-06, No Iterations 2
smoothSolver:  Solving for Uy, Initial residual = 0.0613217, Final residual = 2.54956e-06, No Iterations 3
smoothSolver:  Solving for Uz, Initial residual = 0.060156, Final residual = 2.57411e-06, No Iterations 3
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 4.92767e-05
GAMG:  Solving for p, Initial residual = 0.851138, Final residual = 0.0738231, No Iterations 1
time step continuity errors : sum local = 0.000101196, global = -1.0343e-19, cumulative = -2.17328e-19
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 4.9037e-05
GAMG:  Solving for p, Initial residual = 0.133949, Final residual = 8.9818e-07, No Iterations 6
time step continuity errors : sum local = 9.48158e-10, global = 2.38788e-20, cumulative = -1.93449e-19
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 4.92556e-05
Hello, World Python Pybyind..!
['/home/hsharma/WORK/Project_OFTFCoupling/OF_Pybind11/Dynamic_Smagorinsky/Py_Modules', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python37.zip', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7/lib-dynload', '/home/hsharma/.local/lib/python3.7/site-packages', '/home/hsharma/PACKAGES/Py_TF_HVD/lib/python3.7/site-packages', '.']
0  Foam::error::printStack(Foam::Ostream&) at ??:?
#1  Foam::sigSegv::sigHandler(int) at ??:?
#2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
#3  ? at /io/numpy/numpy/core/src/multiarray/refcount.c:40 (discriminator 1)
#4  ? at /io/numpy/numpy/core/src/multiarray/scalarapi.c:335
#5  ? at /io/numpy/numpy/core/src/multiarray/scalartypes.c.src:351
#6  PyObject_IsTrue in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#7  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#8  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#9  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#10  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#11  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#12  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#13  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#14  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#15  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#16  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#17  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#18  PyEval_EvalCodeEx in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#19  PyEval_EvalCode in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#20  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#21  _PyMethodDef_RawFastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#22  _PyCFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#23  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#24  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#25  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#26  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#27  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#28  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#29  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#30  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#31  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#32  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#33  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#34  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#35  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#36  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#37  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#38  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#39  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#40  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#41  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#42  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#43  PyCFunction_Call in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#44  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#45  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#46  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#47  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#48  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#49  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#50  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#51  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#52  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#53  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#54  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#55  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#56  PyEval_EvalCodeEx in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#57  PyEval_EvalCode in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#58  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#59  _PyMethodDef_RawFastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#60  _PyCFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#61  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#62  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#63  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#64  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#65  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#66  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#67  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#68  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#69  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#70  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#71  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#72  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#73  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#74  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#75  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#76  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#77  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#78  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#79  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#80  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#81  PyCFunction_Call in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#82  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#83  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#84  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#85  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#86  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#87  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#88  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#89  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#90  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#91  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#92  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#93  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#94  PyEval_EvalCodeEx in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#95  PyEval_EvalCode in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#96  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#97  _PyMethodDef_RawFastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#98  _PyCFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#99  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#100  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#101  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#102  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#103  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#104  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#105  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#106  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#107  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#108  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#109  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#110  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#111  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#112  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#113  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#114  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#115  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#116  _PyObject_CallMethodIdObjArgs in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#117  PyImport_ImportModuleLevelObject in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#118  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#119  PyCFunction_Call in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#120  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#121  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#122  _PyFunction_FastCallKeywords in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#123  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#124  _PyEval_EvalFrameDefault in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#125  _PyEval_EvalCodeWithName in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#126  _PyFunction_FastCallDict in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"
#127  ? in "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0"

Reproducible example code

The code to reproduce will need dependencies but this how I am calling the pybind as a function, this function is then called at run time. The purpose is to call the a python module defined function every time-step of the solver. As you can see from the above output that everything works for the first step, in the second step the pybind_call did return the print sys.attr("path")) but fails in after that.

template<class BasicTurbulenceModel>
int  LES_Pybind<BasicTurbulenceModel>::Pybind_call()
{
    int i_1 = 10 ;
    int i_2 = 20 ;


    py::scoped_interpreter guard{}; // start the interpreter and keep it alive


    // This method will make a copy of the C++ defined variable
    // there is way to buy pass
    py::object K = py::cast(i_1);
    py::object K_ = py::cast(i_2);


    //py::print(K);

    py::print("Hello, World Python Pybyind..!"); // use the Python API

    // Load the Module from outside

    // Pass the Array to Python Interpreter
    py::module sys = py::module::import("sys");

    py::exec(R"(import sys;sys.path.insert(0,'path/to/the/python/module/Py_Modules'))");
    py::print(sys.attr("path"));


    py::module calc = py::module::import("calc_test");

    py::object result = calc.attr("add")(1,2);
    py::object version = calc.attr("print_Tf")();

    int n = result.cast<int>();
    auto version_ = version.cast<std::string>();
    Info << "Python Result :" << n  << '\n';
    Info << "Python Result :" << version_ << '\n';
   
}

Any debugging or any kind of inputs are really appreciated. Thank you.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report names no pybind11 repository file or test; begin by reproducing the embedded Python/OpenFOAM loop and inspect the failing stack entries in numpy's refcount.c, scalarapi.c, and scalartypes.c.src. Compare the first and second iterations and use a debugger to identify the invalid object or lifetime involved. Done means the reproduction no longer segfaults across repeated iterations with a regression test or clear reproduction result.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, numpy, python
Domain
backend-api-design, devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.