facebookresearch / facebookresearch/fairscale
Support free-threading Python wheel distributions
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 293
- PR merge metrics
- No merged PRs in 30d
Description
As some of you might be aware, since CPython 3.13, there is experimental support for an alternative interpreter distribution (via [PEP703](https://peps.python.org/pep-0703/)) that disables the GIL altogether, thus enabling support for real thread parallelism.
Since PyTorch now supports free-threaded wheel distributions (https://github.com/pytorch/pytorch/issues/130249), it should be easy producing wheels for fairscale as well.
In order to provide free-threaded wheels, there is a need for extensive testing, which can be summarized in performing the following tasks:
- [ ] Audit Python bindings and declare them free-threading compatible (xref https://py-free-threading.github.io/porting/#updating-extension-modules).
- [ ] Run the test suite with pytest-run-parallel to find potential issues, and fix them.
- [ ] Run the test suite under ThreadSanitizer. Should be feasible, since there are no runtime dependencies aside from Cython.
- [ ] Add cp313t-* to CI to build free-threading wheels.
I can volunteer in taking charge of these tasks if necessary
Contributor guide
Research direction
Start with the Python bindings and the free-threading porting guidance linked in the issue, then run the test suite with pytest-run-parallel. Investigate ThreadSanitizer results and the existing CI wheel configuration. Done means the bindings and tests are compatible with free-threading and CI builds cp313t-* wheels.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- build-system, ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100