[BUG]: `type_info` data races in free threaded Python
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 18k
- Forks
- 2.3k
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 10
Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
master
Problem description
There are a few related data races in free threaded Python (3.13t) involving detail::type_info.
The most serious (found by @vfdev-5) is that the std::vector<type_info *> in registered_types_py is populated outside the internals lock, so a thread may incorrectly see an empty vector (or other crashes). Victor has a PR that addresses this race (#5419).
The other data races are because some fields in type_info may be modified after creation:
type_info::implicit_casts: vector is modified when subclasses are definedtype_info::simple_type: field is set to false when a subclass that uses multiple inheritance is defined
Additionally, a few fields are written after the type_info is exposed in registered_types_cpp/registered_types_py -- I think we want to reorder it so that those fields are initialized earlier.
Finally, there are a few fields that I think are mostly okay in typical usage, but I'm not entirely sure:
type_info::implicit_conversions: modified forpy::implicitly_convertible<A, B>()onB'stype_info.type_info::direct_conversions: pointed-to vector modified byPYBIND11_NUMPY_DTYPE(type, ...)type_info::get_buffer/get_buffer_data: modified bydef_buffer()
I think these are okay because they're typically invoked when the type is bound, before the type_info is used by other threads.
Reproducible example code
See test case in https://github.com/pybind/pybind11/pull/5419
Is this a regression? Put the last known working version here if it is.
Not a regression
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with detail::type_info and the fields around include/pybind11/pybind11.h lines 1396-1412, then review the test case and proposed fix in pull request #5419. Trace writes to registered_types_cpp, registered_types_py, implicit_casts, simple_type, and the conversion and buffer fields under free-threaded Python. Done means the reported data races are addressed and the PR's test case passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100