`dask/tests/test_distributed.py::test_to_hdf_scheduler_distributed` flaky

**What happened**:
The tests within `dask/tests/test_distributed.py::test_to_hdf_scheduler_distributed` sometimes fail in GitHub Actions, for example [here](https://github.com/dask/dask/runs/8170654687?check_suite_focus=true) and [here](https://github.com/dask/dask/runs/8172357478?check_suite_focus=true).

**What you expected to happen**:
The tests to pass every time. They pass locally and in most CI runs.

**Environment**:

- Dask version: `2022.9.0`
- Python version: `3.9`
- Operating System: `macOS`
- Install method (conda, pip, source): `conda`