[eudsl-llvmpy] Free-threaded data races in unsynchronized static registries (casterMap + Python-pass registries)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 79
- Forks
- 14
- Avg merge
- 11h 43m
- Merged PRs (30d)
- 72
Description
## Summary
`eudsl-llvmpy` builds its nanobind extension with `FREE_THREADED` (`Py_MOD_GIL_NOT_USED`, see `projects/eudsl-llvmpy/CMakeLists.txt:106`), so it can be imported under a free-threaded (no-GIL) CPython. Several process-wide caches are plain **unsynchronized** `std::unordered_map`s. Concurrent writes and reads on them from multiple Python threads are data races.
This is flagged (not urgent): typical usage is **register/populate at import, then run**, so writes and reads don't actually overlap in practice, and the whole extension currently relies on `nb::gil_scoped_acquire` in the hot callbacks. But once we care about genuine multi-threaded use under the free-threaded build, these should be made safe together.
## Affected registries
1. **Python-pass registries** (added in the Python-IR-passes stack, `src/IR/Passes.cpp`):
```cpp
std::unordered_map &pythonModulePassRegistry();
std::unordered_map &pythonFunctionPassRegistry();
```
`register_python_pass(...)` writes; the `PassBuilder::registerPipelineParsingCallback` lambdas read them during `run_passes(...)`. Concurrent `register_python_pass` (write) + `run_passes` (read) races.
Interim mitigation already in place: the `register_python_pass` docstring tells callers to register from a single thread (e.g. at import) before running pipelines concurrently.
2. **Caster registry** (`src/IR/Casters.cpp:20`):
```cpp
std::unordered_map &casterMap();
```
Written by the caster-registration path, read on every value cast. Same shape of race. The Python-pass registries were intentionally modeled on this existing/accepted pattern.
## Possible fixes (to decide later)
- Wrap each registry in a small accessor guarded by a `std::mutex` (or `std::shared_mutex` for read-mostly access), or
- Document + enforce a "populate-before-use, then immutable" contract (freeze after import), or
- Use a concurrent map / copy-on-write snapshot for the read-heavy lookup paths.
Whatever we pick, it should be applied consistently to **both** `casterMap()` and the two Python-pass registries, since they share the pattern.
## References
- `projects/eudsl-llvmpy/src/IR/Passes.cpp` — `pythonModulePassRegistry()` / `pythonFunctionPassRegistry()` and the parsing-callback readers
- `projects/eudsl-llvmpy/src/IR/Casters.cpp:20` — `casterMap()`
- `projects/eudsl-llvmpy/CMakeLists.txt:106` — `FREE_THREADED` build
- Surfaced during review of #605 (named Python-pass registration).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with projects/eudsl-llvmpy/CMakeLists.txt:106, then inspect the registries and their readers in src/IR/Passes.cpp and src/IR/Casters.cpp:20. Compare registration and lookup paths, choose a consistent synchronization or immutability strategy for all three registries, and verify that concurrent registration and use no longer race.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100