[SUREFIRE-726] Test list preprocessor support for tests to be run

**[Janmejay Singh](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=janmejay)** opened **[SUREFIRE-726](https://issues.apache.org/jira/browse/SUREFIRE-726?redirect=false)** and commented

This exposes an interface(named TestListPreprocessor, which has a method named preprocessTestClasses) that can be implemented by a user and injected into surefire plugin configuration to have it used for preprocessing list of tests to be run. Given the original test list, user's preprocessing algorithm can choose to prune or reorder the list and return it back from TestListPreprocessor#preprocessTestClasses, which is then used as the list of tests to be executed.

The patches attached expose a configuration element named "testPreprocessor" the default value of which is "none". User can choose to set it to a value matching the format "\[::]" and have the class loaded and called with list of tests to be run. The list of tests returned by the call is then considered for execution.

Patch description:
The feature is done in 2 patches. Description follows:

0001-Adds-test-list-preprocessor-support-which-loads-an-o.patch
This patch actually adds the feature and unit tests. It adds the interface, an abstraction that encapsulates aforementioned configuration, booter serialization/deserialization, directory-scanner changes etc. The changes have effect in both forked and in-process execution mode.

0002-Integration-test-for-preprocessor.-2-modules-one-pro.patch
This patch adds an integration test for preprocessor feature. The integration test uses a 2 module setup, where the first module implements the TestListPreprocessor interface which selects only even indexed test classes from the list passed in. The second module(which depends on first) uses the artifact from first and uses the following configuration:

---

\
\test.preprocessor.EvenTestOnlyPreprocessor[org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT]\
\

---

The test then asserts that only 2(0th and 2nd) of the 4 test classes the second module has are executed.

The patches are created on a git-svn clone using format-patch, please use 'patch -p1 < 0001-Adds...'(while in the trunk directory) to apply the patches on svn repository. If using git-svn, git-am can be directly be invoked.

The patches are done over http://svn.apache.org/repos/asf/maven/surefire/trunk@1091357 (trunk HEAD) so should apply cleanly on any recent enough revision.

---

Context:
http://stackoverflow.com/questions/5124823/reducing-the-build-time-hudson
http://code.google.com/p/tlb/issues/detail?id=1
http://test-load-balancer.github.com

---

**Affects:** 2.9

**Attachments:**
- [0001-Adds-test-list-preprocessor-support-which-loads-an-o.patch](https://issues.apache.org/jira/secure/attachment/12720019/0001-Adds-test-list-preprocessor-support-which-loads-an-o.patch) (_53.33 kB_)
- [0002-Integration-test-for-preprocessor.-2-modules-one-pro.patch](https://issues.apache.org/jira/secure/attachment/12720194/0002-Integration-test-for-preprocessor.-2-modules-one-pro.patch) (_17.19 kB_)
- [0003-exposed-testPreprocessor-configuration-parameter-as-.patch](https://issues.apache.org/jira/secure/attachment/12720129/0003-exposed-testPreprocessor-configuration-parameter-as-.patch) (_31.00 kB_)
- [0004-exposed-testPreprocessor-for-IntegrationTestMojo-as-.patch](https://issues.apache.org/jira/secure/attachment/12720022/0004-exposed-testPreprocessor-for-IntegrationTestMojo-as-.patch) (_1.08 kB_)

**Issue Links:**
- [SUREFIRE-1836](https://issues.apache.org/jira/browse/SUREFIRE-1836) re-run should specify alternatives to the forking model

- [SUREFIRE-701](https://issues.apache.org/jira/browse/SUREFIRE-701) showSuccess option for test reports: show reports only for failed tests

- [SUREFIRE-713](https://issues.apache.org/jira/browse/SUREFIRE-713) Allow setting of excludes and includes via text files

- [SUREFIRE-131](https://issues.apache.org/jira/browse/SUREFIRE-131) Excluding tests with command line pattern

- [SUREFIRE-526](https://issues.apache.org/jira/browse/SUREFIRE-526) Better support for other plugins to determine which tests are included/excluded and the order the tests get run

- [SUREFIRE-528](https://issues.apache.org/jira/browse/SUREFIRE-528) Splitting tests names in two categories : success and failure

- [SUREFIRE-530](https://issues.apache.org/jira/browse/SUREFIRE-530) Allow runtime ordering of tests to be specified

- [SUREFIRE-561](https://issues.apache.org/jira/browse/SUREFIRE-561) after running test, when tests fail, it's hard to the find the failure reason

- [SUREFIRE-632](https://issues.apache.org/jira/browse/SUREFIRE-632) add a test-failed goal, to only run the tests which failed last time

- [SUREFIRE-582](https://issues.apache.org/jira/browse/SUREFIRE-582) 1. Define "set of tests" that should be run at the begining. 2. Fail build if any error/failure occured in some specified "set of tests".

- [SUREFIRE-528](https://issues.apache.org/jira/browse/SUREFIRE-528) Splitting tests names in two categories : success and failure
(_**"is depended upon by"**_)

1 votes, 4 watchers

Start by reviewing the attached feature and integration-test patches, then trace the TestListPreprocessor interface and the testPreprocessor configuration. Check the integration scenario where EvenTestOnlyPreprocessor selects two of four test classes, and verify the behavior in both forked and in-process execution modes.

Written by the indexing model from the issue text.