lightly-ai / lightly-ai/lightly
Proper distributed tests
@saud5150 is already working on this.
Since Jul 20, 2026.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 367
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 5
Description
We do some proper distributed tests in `tests/utils/test_dist__gather__losses.py`, however those are separate from the test modules of the individual losses so adding new losses always gets forgotten or we try to mock the distributed behaviour, which often does not work very well, e.g. in #1980 .
Instead, it would be nice to have a shared distributed context that we could submit distributed tests to. This should run entirely on CPU (gloo backend for PyTorch). A really nice example for this is [`torchmetrics`](https://github.com/Lightning-AI/torchmetrics/blob/master/tests/unittests/conftest.py).
Let me know if you'd like to work on this and I will give some more proper guidance and detail on how this can be done.
Following things to be done:
- [ ] add distributed pool to the test setup following torchmetrics (test with a single loss)
- [ ] add to CI
- [ ] abstract it nicely
- [ ] expand coverage to other losses/use cases
- [ ] get rid of `tests/utils/test_dist__gather__losses.py`
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.