GENERATE inside of a SECTION stops generating after the first failure
- Dominant language
- C++
- Stars
- 21.5k
- Forks
- 3.5k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 2
Description
**Describe the bug**
`GENERATE` behaves similarly to a `SECTION` - the code after the `GENERATE` is run multiple times, with a different value each time. All cases should be run at all times, even if some of them are failing. This works as expected if the `GENERATE` is directly in a `TEST_CASE`, but not if it is inside of a `SECTION`.
**Expected behavior**
`GENERATE` should behave the same way inside and outside of a `SECTION`.
**Reproduction steps**
Source:
```c++
// This is OK
TEST_CASE("GENERATE should behave as section") {
int i = GENERATE(0,1,2,3);
// should generate 2 passing and 2 failing sections
REQUIRE(i % 2 == 0);
}
// This only runs for 0, 1. Variants for 2, 3 are not run.
TEST_CASE("GENERATE should behave as section - nested in SECTION") {
SECTION("a section") {
int i = GENERATE(0,1,2,3);
// should generate 2 passing and 2 failing sections
REQUIRE(i % 2 == 0);
}
}
```
Output:
```
# test case 1 - ok
test.cpp:7: passed: i % 2 == 0 for: 0 == 0
test.cpp:7: failed: i % 2 == 0 for: 1 == 0
test.cpp:7: passed: i % 2 == 0 for: 0 == 0
test.cpp:7: failed: i % 2 == 0 for: 1 == 0
# test case 2 - not ok
test.cpp:14: passed: i % 2 == 0 for: 0 == 0
test.cpp:14: failed: i % 2 == 0 for: 1 == 0
```
**Platform information:**
- OS: Linux
- Compiler+version: **GCC 10.3.0**
- Catch version: **v2.13.6**, devel (c77ba5314a82b468f9591b7ec7e90ada67a7c664)
**Additional context**
With `CHECK` instead of `REQUIRE`, it works as expected.
Contributor guide
Assessment
This issue has not been assessed yet.