[BUG]: Document MSVC `std::mutex` ABI hazard with older `msvcp140.dll` and `_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR`
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 18k
- Forks
- 2.3k
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 10
Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
3.0.4
Problem description
We recently debugged a real downstream Windows crash where a pybind11-based extension was built with newer MSVC STL headers, but at runtime its std::mutex lock path resolved to an older msvcp140.dll that had been preloaded by another Python wheel.
Microsoft documents _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR as the compatibility workaround for this newer-toolset / older-redistributable mismatch. It would be useful for pybind11’s Windows build documentation to mention this for projects distributing MSVC-built binary wheels.
The hazard
With newer VS 2022 MSVC STL headers, std::mutex can use a constexpr zero-initialized layout. The constructor may emit no runtime _Mtx_init_in_situ call; later lock operations call _Mtx_lock from msvcp140.dll.
If another wheel in the same Python process has already loaded an older msvcp140.dll under the normal DLL name, the Windows loader may bind the extension’s _Mtx_lock import to that older runtime. The older runtime may expect the pre-constexpr mutex layout, so it can interpret the newer zero-initialized object incorrectly and crash with an access violation.
In the observed case, the first affected lock happened inside pybind11 import-time internals initialization:
PYBIND11_MODULE_PYINIT
-> pybind11::detail::ensure_internals()
-> pybind11::detail::get_internals()
-> internals_pp_manager::create_pp_content_once()
-> std::lock_guard<std::mutex> lock(pp_set_mutex_)
But the issue class is broader than that one lock: any std::mutex compiled into the extension can be affected if construction and locking are interpreted by incompatible MSVC STL runtime layouts.
Real-world incident
- Incident: metaopt/optree#278
- Downstream fix: metaopt/optree#279
The crash dump showed:
msvcp140!mtx_do_lock+0x74
optree._C!PyInit__C+0xa098
optree._C+0x4e29
optree._C!PyInit__C+0x3d
The faulting address was inside pyarrow’s bundled msvcp140.dll 14.28.29334.0. The affected wheel’s import table showed _Mtx_lock / _Mtx_unlock, but not _Mtx_init_in_situ.
After defining _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR for the extension target, the built .pyd import table included:
_Mtx_lock
_Mtx_init_in_situ
_Mtx_unlock
The original reporter confirmed that the patched wheel imports cleanly in the same environment without the previous import-order workaround.
Suggested documentation text
A note like this could live in the Windows / compiling docs:
MSVC
std::mutexcompatibility with oldermsvcp140.dllWindows extension modules built with newer MSVC STL headers and dynamic CRT linking may be loaded into Python processes where another wheel has already loaded an older
msvcp140.dll.If the extension uses
std::mutex, this can produce an MSVC STL mutex ABI mismatch: the extension’s mutex object is constructed using the newer header layout, but_Mtx_lockresolves at runtime to an oldermsvcp140.dllimplementation that expects the older layout. One symptom is an access violation inmsvcp140!mtx_do_lockduring module import.Microsoft provides
_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTORas a compatibility workaround. Projects affected by this can define it for their extension target:if(MSVC) target_compile_definitions(your_module PRIVATE _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR) endif()For setuptools:
import sys from setuptools import Extension ext = Extension( 'your_module', sources=[...], define_macros=( [('_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR', None)] if sys.platform == 'win32' else [] ), )You can verify that the macro affected the build by inspecting the
.pydimport table:llvm-objdump -p your_module.cp*-win_amd64.pyd | grep _Mtx_Expected output includes
_Mtx_init_in_situalongside_Mtx_lock/_Mtx_unlock._Mtx_destroy_in_situis not expected.Trade-offs: mutex construction is no longer a constexpr no-op, and
constinit std::mutexpatterns are disabled inside that target. For typical extension modules this is usually negligible, but projects should apply the macro at the extension-target level rather than globally.
Reproducible example code
Is this a regression? Put the last known working version here if it is.
Not a regression
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Windows or compiling documentation and review the suggested MSVC std::mutex compatibility text in this issue. Document the _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR workaround for affected extension targets, including the CMake and setuptools examples, and explain verification with llvm-objdump; done when the guidance covers the hazard, trade-offs, and expected import-table symbols.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp, python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100