opensearch-project / opensearch-project/OpenSearch

[Feature Request] Forbid the use of Thread.sleep in tests

Open
#20,004 2 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Build enhancement
Dominant language
Java
Stars
13.7k
Forks
3k
Avg merge
2d 23h
Merged PRs (30d)
108

Description

Is your feature request related to a problem? Please describe

Using a fixed sleep in tests is a great way to introduce a flaky test. It's almost always better to poll for the condition you're waiting for rather than waiting a fixed amount of time. Here are a handful of flaky tests from the past that were caused by fixed sleeps:

  • #19422
  • #14288
  • #18820

This list is certainly not exhaustive.

Describe the solution you'd like

Use forbidden APIs to forbid the usage of Thread#sleep in test code. There are certainly cases where sleeping will still be needed so this can be overridden with the @SuppressForbidden annotation, but making the default behavior to fail the build will make it clear to contributors and reviewers that any usage of sleep needs scrutiny.

Additional context

Because usage of Thread.sleep is currently quite pervasive, my rollout plan for this is to chip away one at a time by making it an optional property that can be turned on one test suite at a time. For example, the following can be added to server/build.gradle to enable this for internal cluster tests:

tasks.named('forbiddenApisInternalClusterTest').configure { forbidSleep() }

This will avoid a huge PR to annotate or remove hundreds of usages across all the projects. Once we reach a critical mass we can switch the default in this repo.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with server/build.gradle and the forbiddenApisInternalClusterTest configuration, then trace how forbidSleep() is enabled for a test suite. Done means enabled suites reject Thread.sleep usage by default, while cases marked with @SuppressForbidden remain possible and rollout can proceed one suite at a time.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.