Flakiness in test_shuffle.py
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Several tests in test_shuffle.py are very flaky.
If I change `.github/workflows/tests.yaml` as follows, to rerun the tests 20 times (ci1 + not ci1) per environment:
```
pytest distributed/shuffle/tests/test_shuffle.py --count=10 --runslow \
--leaks=...
```
I get the following failure rates:
| test | n. failures |
| --- | --- |
| distributed/shuffle/tests/test_shuffle.py::test_clean_after_close | 1 |
| distributed/shuffle/tests/test_shuffle.py::test_closed_input_only_worker_during_transfer | 1 |
| distributed/shuffle/tests/test_shuffle.py::test_closed_worker_during_transfer | 29 |
| distributed/shuffle/tests/test_shuffle.py::test_crashed_worker_during_transfer | 6 |
| distributed/shuffle/tests/test_shuffle.py::test_restarting_during_transfer_raises_killed_worker | 38 |
Additionally, `test_crashed_worker_during_transfer` deadlocks in a way that's irrecoverable on Windows, causing the whole test suite to be killed by
https://github.com/dask/distributed/blob/eb297b3e7f4d045f80be0a5b50b42a579adc37ee/pyproject.toml#L155-L162
logs: https://github.com/crusaderky/distributed/actions/runs/5761255813
CC @hendrikmakait
Contributor guide
Assessment
This issue has not been assessed yet.