pybind / pybind/pybind11

[BUG]: Crash if 2 modules have conflicting module-local types

Open
#4,097 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
Problem description

When two PyBind11 modules define different, ABI-incompatible classes with the same name, it's possible to pass either one of them to a function expecting the other, resulting in memory unsafety and crashes.

Reproducible example code

testcat.cpp

#include <pybind11/pybind11.h>
#include <iostream>

namespace py = pybind11;

struct Pet {
    char *name;
    const char *getName() const { return name; }
};

Pet createPet() {
    return Pet { "cat" };
}

void printPetName(const Pet& pet) {
    std::cout << "pet name: " << pet.getName() << std::endl;
}

PYBIND11_MODULE(testcat, m) {
    m.def("create_pet", &createPet);
    m.def("print_pet_name", &printPetName);
    py::class_<Pet>(m, "Pet", py::module_local())
        .def("get_name", &Pet::getName);
}

testdog.cpp

#include <pybind11/pybind11.h>
#include <iostream>

namespace py = pybind11;

struct Pet {
    std::string name;
    std::string getName() const { return name; }
};

Pet createPet() {
    return Pet { "dog" };
}

void printPetName(const Pet& pet) {
    std::cout << "pet name: " << pet.getName() << std::endl;
}

PYBIND11_MODULE(testdog, m) {
    m.def("create_pet", &createPet);
    m.def("print_pet_name", &printPetName);
    py::class_<Pet>(m, "Pet", py::module_local())
        .def("get_name", &Pet::getName);
}

setup.py

from pybind11.setup_helpers import Pybind11Extension
from setuptools import setup

ext_modules = [
    Pybind11Extension("testdog", ["testdog.cpp"]),
    Pybind11Extension("testcat", ["testcat.cpp"]),
]

setup(name="pettest", ext_modules=ext_modules)

pyproject.toml

[build-system]
requires = ["setuptools", "wheel", "pybind11"]
build-backend = "setuptools.build_meta"

crash.py

import testcat
import testdog
testdog.print_pet_name(testcat.create_pet())

Results

Dropping all of those files into a directory and then running:

$ pip install .
$ python crash.py

results in a segmentation fault, due to incorrectly treating a Pet from testcat.so as though it's a Pet from testdog.so, rather than a distinct type that happens to have the same name.

This appears to have been an intentional feature introduced by #1007, but application of this feature across different modules built at different times against different (potentially ABI incompatible) libraries can result in memory unsafety and undefined behavior.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the crash with testcat.cpp, testdog.cpp, setup.py, and crash.py, then inspect the module-local type behavior introduced by #1007. The fix is done when ABI-incompatible Pet types from separate modules cannot be passed interchangeably, eliminating the segmentation fault without losing distinct module-local types.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.