apache / apache/maven-surefire

How to utilize all sessions if tests are run across multiple xmls

Open
#842 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

### New feature, improvement proposal

hello

we use testNG to run tests via suites, where each suite denotes the functionality of each team, e.g. team1, team2, team3

The problem is that all suites have different qty of tests. because of this the free quota (selenium grid, moon etc) in the cloud is idle. how to maximize the use of free sessions?

For example, we run 3 suites in parallel. two suites have 10 tests each, the third suite has 20 tests.
(suppose the total free quota is 15 sessions.)
The settings are as follows:

`

org.apache.maven.plugins
maven-surefire-plugin
${maven-surefire-plugin.version}


-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"


team1.xml
team2.xml
team3.xml

tests
5



suitethreadpoolsize
3





`

result

![Image](https://github.com/user-attachments/assets/60d140ea-21af-4e48-b92f-4dfffe15f601)

![Image](https://github.com/user-attachments/assets/f6214a55-a12a-4b96-b259-253d5b849ea4)

the third suite used 5 sessions. 10 free sessions were not used.
(I would like the remaining 10 tests in suite 3 to be executed in 10 threads in parallel)

second example

config

`

org.apache.maven.plugins
maven-surefire-plugin
${maven-surefire-plugin.version}


-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"


team1.xml
team2.xml
team3.xml

tests
15


`

![Image](https://github.com/user-attachments/assets/f9eca101-8fed-474a-9381-bd8569efb8fd)

since in the second example suites are run sequentially, there are 5 sessions free while suites 1 and 2 are running.

It is possible to run all suites in parallel and have all tests/classes in them use a global number/limit of threads?

is it possible to change the number of threads (threadCount) for each suites dynamically ?

here real example from our production

![Image](https://github.com/user-attachments/assets/66185ebe-1eb9-4c5d-a055-d126bb31fedb)

number 1 -- a suite with thread-count=1 and minimum number of tests. it ended before the others. the free session left after it is idle.

number 2 -- similar.

number 3 -- the suite that runs the longest of all suites.

The vertical purple line is the ideal time spent on tests, assuming that the entire quota (20 in this case) is used.

code:
suites:
https://gist.github.com/VZEROALL/27acc1216f0be7f0e481546f399e4827
https://gist.github.com/VZEROALL/6ec11b39b2d52227d219bccb7137e801
https://gist.github.com/VZEROALL/a7332da802abba308d9288a44a46bc42

test class:
https://gist.github.com/VZEROALL/5dc21dd24fc7b45f282f7a3c35696304

example1 config
https://gist.github.com/VZEROALL/187118b93ce6464840e4066b4df8c2e3
example2 config
https://gist.github.com/VZEROALL/ec33f2f3cd500c9c582f433918f9634a

Contributor guide

Open the contributing guide

Research direction

Start with the Maven Surefire configuration and the supplied suite XML and example configuration gists, focusing on parallel, threadCount, and suiteThreadPoolSize. Determine whether a global thread limit across the three suites and dynamic per-suite allocation can be specified, then add coverage for the requested scheduling behavior if the project supports it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.