pybind / pybind/pybind11

[BUG]: `type_info` data races in free threaded Python

Open
#5,421 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

master

Problem description

There are a few related data races in free threaded Python (3.13t) involving detail::type_info.

The most serious (found by @vfdev-5) is that the std::vector<type_info *> in registered_types_py is populated outside the internals lock, so a thread may incorrectly see an empty vector (or other crashes). Victor has a PR that addresses this race (#5419).

The other data races are because some fields in type_info may be modified after creation:

  • type_info::implicit_casts: vector is modified when subclasses are defined
  • type_info::simple_type: field is set to false when a subclass that uses multiple inheritance is defined

Additionally, a few fields are written after the type_info is exposed in registered_types_cpp/registered_types_py -- I think we want to reorder it so that those fields are initialized earlier.

Finally, there are a few fields that I think are mostly okay in typical usage, but I'm not entirely sure:

  • type_info::implicit_conversions: modified for py::implicitly_convertible<A, B>() on B's type_info.
  • type_info::direct_conversions: pointed-to vector modified by PYBIND11_NUMPY_DTYPE(type, ...)
  • type_info::get_buffer/get_buffer_data: modified by def_buffer()

I think these are okay because they're typically invoked when the type is bound, before the type_info is used by other threads.

Reproducible example code
See test case in https://github.com/pybind/pybind11/pull/5419
Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with detail::type_info and the fields around include/pybind11/pybind11.h lines 1396-1412, then review the test case and proposed fix in pull request #5419. Trace writes to registered_types_cpp, registered_types_py, implicit_casts, simple_type, and the conversion and buffer fields under free-threaded Python. Done means the reported data races are addressed and the PR's test case passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.