apache / apache/maven-surefire
[SUREFIRE-1477] CI support for parallel test execution
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
**[Stephen Connolly](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stephenc)** opened **[SUREFIRE-1477](https://issues.apache.org/jira/browse/SUREFIRE-1477?redirect=false)** and commented
CI build servers often want to use multiple compute resources to reduce build times in exchange for additional CPU costs.
One of the biggest consumers of build time is test execution, so much so that people are often willing to run multiple builds in parallel on different machines in order to get the total testing wall clock time lower.
For example: https://wiki.jenkins.io/display/JENKINS/Parallel+Test+Executor+Plugin
It would be nice if surefire made all this easier and more CI system agnostic.
We already have some of this available: https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#runOrder has the option `balanced` that will use a statistics file `.surefire-XXXXXXXXX` to distribute tests between different parallel worker threads.
What is needed to make this better is the ability to use the `balanced` run ordering across Maven JVMs.
If we had two system properties:
* `surefire.mvnCount`
* `surefire.mvnOffset`
Then surefire could use the `balanced` run order to deterministically decide the ordered list of tests... it would then start at `mvnOffset` and run every `mvnCount`th test.
CI users could then run three maven builds in parallel:
* `mvn -Dsurefire.mvnCount=3 -Dsurefire.mvnOffset=0 test`
* `mvn -Dsurefire.mvnCount=3 -Dsurefire.mvnOffset=1 test`
* `mvn -Dsurefire.mvnCount=3 -Dsurefire.mvnOffset=2 test`
This would ensure that all the tests are run once, but the wall clock time should be best case approx 1/3th of the wall clock for the full build time
---
**Issue Links:**
- [SUREFIRE-1529](https://issues.apache.org/jira/browse/SUREFIRE-1529) Implement test bucketing
(_**"is duplicated by"**_)
Contributor guide
Research direction
Start by reading the linked SUREFIRE-1529 issue, which is marked as a duplicate, and then inspect the existing balanced run-order behavior described in the Maven Surefire documentation. Use the three example Maven commands as the intended CI scenarios; done means tests are deterministically distributed across Maven JVMs without omissions or overlap.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, ci-cd, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100