Split tests for parallelization
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 4k
- Forks
- 773
- PR merge metrics
- No merged PRs in 30d
Description
Recently, I've seen a lot of interest in and discussion on running JS tests in parallel. Most of these rely on faking concurrency through async behavior, while that is definitely an interesting approach I am not convinced it is the proper solution for browser-based tests that often rely on the document and other global/shared state. That said, I think finding a way to parallelize tests would be hugely beneficial, especially for larger code bases.
I'd like to propose that QUnit bake in support for splitting tests into groups at runtime. This would then allow users to parallelize test runs by using multiple, independent instances of their test page.
I imagine this could be based on url params like so:
/tests.html?batches=10&batch=1
This would split the tests into 10 different batches and then run the first batch. You can then adjust the batch parameter to run the other groups in other browser instances/tabs.
As an initial implementation, this could be super simple and just break the tests up into equally sized batches. Then, as an enhancement, we could standardize a test output that can be fed back into QUnit to allow weighted splitting to ensure the batches run in approximately equal amounts of time.
The primary benefit of this would be to allow parallelization, but in doing so it would also help identify non-atomic tests in the same way as the recently introduced test randomization feature does.
For full disclosure, this would benefit a recently implemented Testem feature to support running multiple test pages in parallel.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named in the issue. Start from QUnit's runtime test scheduling and browser test-page entry point; completion means URL parameters select equally sized batches that can run independently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100