Cut CI wall clock: rebalance the Spark test jobs and surface the slow tail
Open
area:ci
priority:high
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
The critical path for a PR is `test-spark-java17-java-tests-part1` at roughly 70 minutes, and the part1/part2/part3 splits follow module boundaries rather than measured duration.
- Rebalance the splits by measured job duration, and re-measure whenever what those jobs run changes.
- Report the slowest tests per job so the long tail is visible and can be fixed or moved off the PR path.
- Give contributors early signal: run the cheap validations and a fast compile before the long test jobs, so an obviously broken PR fails in minutes rather than after an hour.
Part of #19524.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.