dask / dask/distributed

Introduce public-facing `distributed.testing` module and deprecate `distributed.utils_test`

Open
#6,897 0 comments 0 reactions 0 assignees View on GitHub
enhancement tests
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In #6806, we have reached support for the idea of introducing a public-facing `distributed.testing` module that contains all the testing utilities that we would like friends-of-dask and users to use in their CI. `distributed.utils_test` should then be deprecated in favor of `distributed.testing` issuing a deprecation warning pointing toward `distributed.testing` whenever someone imports `distributed.utils_test`. All imports from `utils_test` must still work after the restructuring. Whenever possible, we should add a more pointed deprecation warning to utilities that points toward a preferred alternative.

The contents of `distributed.utils_test` should be moved into a private module, e.g. `distributed._testing`. From there, we import a _subset_ of the testing utilities into `distributed.testing` to be exposed as public-facing utilities. These utilities should be well-documented and represent the best practices we follow for testing `distributed` that we encourage others to use in CI.
We want to highlight that these utilities are not considered stable with clear deprecation cycles, but
> when we make a change:
> * it will it will probably cause your test suite to fail
> * but any failure we cause should be actionable
> * and we should come help you fix it
> * and most importantly you should always be able to copy paste an old version to unblock any PRs and issues with higher priority
>
> (https://github.com/dask/distributed/issues/6806#issuecomment-1205477913)

and may be changed by us to enforce best practices, but set the expectation that we will support the Dask ecosystem in adapting to these changes through documentation or active development. The private contents of `distributed.utils_test` should only be used within `distributed` and can therefore be changed without affecting anybody in the Dask ecosystem.

When it comes to the concrete contents of `distributed.testing`, we have the following guidelines:
* Do not export fixtures, use context managers instead.
* Less is more.
* Document properly.

After a first search of friends-of-dask, I have found the following utilities in use:
* loop (8x)
* cluster (5x)
* gen_cluster (5x)
* loop_in_thread (4x)
* cleanup (3x)
* popen (2x)
* captured_logger (2x)
* inc (2x)
* client (1x)
* cluster_fixture (1x)
* gen_test (1x)
* raises_with_cause (1x)
* slowinc (1x)
* varying (1x)
* `import utils_test` (1x)

Consequently, I think `loop`, `cluster` and `gen_cluster` may be good candidates to start with in `distributed.testing`. For the remainder, we should take a look at the tests in question to see if we should rewrite the test, copy+paste the utility, or include it in `distributed.testing`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.