opensearch-project / opensearch-project/OpenSearch
[Feature Request] Forbid the use of Thread.sleep in tests
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 13.7k
- Forks
- 3k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 108
Description
Is your feature request related to a problem? Please describe
Using a fixed sleep in tests is a great way to introduce a flaky test. It's almost always better to poll for the condition you're waiting for rather than waiting a fixed amount of time. Here are a handful of flaky tests from the past that were caused by fixed sleeps:
- #19422
- #14288
- #18820
This list is certainly not exhaustive.
Describe the solution you'd like
Use forbidden APIs to forbid the usage of Thread#sleep in test code. There are certainly cases where sleeping will still be needed so this can be overridden with the @SuppressForbidden annotation, but making the default behavior to fail the build will make it clear to contributors and reviewers that any usage of sleep needs scrutiny.
Additional context
Because usage of Thread.sleep is currently quite pervasive, my rollout plan for this is to chip away one at a time by making it an optional property that can be turned on one test suite at a time. For example, the following can be added to server/build.gradle to enable this for internal cluster tests:
tasks.named('forbiddenApisInternalClusterTest').configure { forbidSleep() }
This will avoid a huge PR to annotate or remove hundreds of usages across all the projects. Once we reach a critical mass we can switch the default in this repo.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with server/build.gradle and the forbiddenApisInternalClusterTest configuration, then trace how forbidSleep() is enabled for a test suite. Done means enabled suites reject Thread.sleep usage by default, while cases marked with @SuppressForbidden remain possible and rollout can proceed one suite at a time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100