automation to help prevent slow tests from being added to the codebase [LUCENE-10257]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
We have thousands and thousands of unit tests.
The problem: when new PRs go in, the new test could take 30 seconds, but all the checks will pass, so it sneaks in. So nobody bothers to look at how long the new test takes.
After a year or so, test runtime tends to double, until the work is done to then fix all these slow long-tail tests to become reasonable. It becomes exhausting.
Can we improve the PR process to prevent these tests from making it in?
I don't know what it would be: more visibility into how long the unit test run took from the github action or something like that? At least maybe more visibility into the "top 10 slowest tests" that the build prints at the end?
I'm not trying to suggest hard limits, as that seems really annoying/frustrating. Just something better.
---
Migrated from [LUCENE-10257](https://issues.apache.org/jira/browse/LUCENE-10257) by Robert Muir (@rmuir)
Contributor guide
Research direction
Start by reviewing the GitHub Action checks and the build output that reports unit-test timing, especially whether a top-10 slowest-tests summary is already available. Define a non-blocking way to expose new or slow test runtimes in pull requests; done means the PR process provides better visibility without imposing hard limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ci-cd, developer-experience, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100