apache / apache/lucene

automation to help prevent slow tests from being added to the codebase [LUCENE-10257]

Open
#11,293 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

We have thousands and thousands of unit tests.

The problem: when new PRs go in, the new test could take 30 seconds, but all the checks will pass, so it sneaks in. So nobody bothers to look at how long the new test takes.

After a year or so, test runtime tends to double, until the work is done to then fix all these slow long-tail tests to become reasonable. It becomes exhausting.

Can we improve the PR process to prevent these tests from making it in?
I don't know what it would be: more visibility into how long the unit test run took from the github action or something like that? At least maybe more visibility into the "top 10 slowest tests" that the build prints at the end?

I'm not trying to suggest hard limits, as that seems really annoying/frustrating. Just something better.

---
Migrated from [LUCENE-10257](https://issues.apache.org/jira/browse/LUCENE-10257) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the GitHub Action checks and the build output that reports unit-test timing, especially whether a top-10 slowest-tests summary is already available. Define a non-blocking way to expose new or slow test runtimes in pull requests; done means the PR process provides better visibility without imposing hard limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ci-cd, developer-experience, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.