catchorg / catchorg/Catch2

Option to run test(s) multiple times with different RNG seeds

Open
#1,475 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

## Description
While working on [a certain project](https://github.com/CleverRaven/Cataclysm-DDA) it happens sometimes that there is some rare test failure and I want to find a particular RNG seed which makes it reproducible. Currently I do this via a command like
```
for i in {1..1000}; do test_program test_name --rng-seed $i || break; done
```
But for this project in particular the global setup code that runs before all the tests is quite expensive (because it's loading lots of game data), so this is rather slow. If it were possible to run the same test many times within a single run of the test program, but with a different RNG seed each time, that would be much faster.

### Extra information
* Catch version: **v2.5.0**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.