catchorg / catchorg/Catch2

Nested GENERATEs cause Catch2 to loop indefinitely

Open
#2,734 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Describe the bug**
Using nested `GENERATE`s inside a `SECTION` together with raw values causes Catch2 to loop indefinitely.

**Expected behavior**
Catch2 should finish without going into an infinite loop.

To be fair, I'm not sure whether this is a supported use case. Perhaps nested GENERATE statements are not supported, or perhaps we are nesting them incorrectly. I haven't been able to find any explicit documentation about this, but if I've missed something, please refer me to it.

**Reproduction steps**
I've managed to reproduce this using the following minimal example:
```cpp
#include

TEST_CASE("Explicitly nested GENERATEs", "[explicit][infinite]")
{
SECTION("Inside section 1")
{
SECTION("Inside section 2")
{
const auto& [str1, str2] = GENERATE(table({
{ "myStr0.1", "myStr0.2" }, // Removing this line lets Catch2 finish
{ GENERATE("myStr1.1", "myStr1.2"), "myStr2.1" }
}));
CHECK(str1 == "myStr1.1");
}
}
}

TEST_CASE("Implicitly nested GENERATEs with copy", "[implicit][copy][fine]")
{
const auto& myStr1 = GENERATE("myStr1.1", "myStr1.2");
const auto& [str1, str2] = GENERATE_COPY(table({
{ "myStr0.1", "myStr0.2" },
{ myStr1, "myStr2.1" }
}));
CHECK(str1 == "myStr1.1");
}

TEST_CASE("Implicitly nested GENERATEs with ref", "[implicit][ref][fine]")
{
const auto& myStr1 = GENERATE("myStr1.1", "myStr1.2");
const auto& [str1, str2] = GENERATE_REF(table({
{ "myStr0.1", "myStr0.2" },
{ myStr1, "myStr2.1" }
}));
CHECK(str1 == "myStr1.1");
}

TEST_CASE("No nested GENERATEs", "[fine]")
{
const auto& [str1, str2] = GENERATE(table({
{ "myStr1.1", "myStr2.1" },
{ "myStr1.2", "myStr2.1" }
}));
CHECK(str1 == "myStr1.1");
}
```

I'm compiling the code using:
```shell
clang++ infinitegenerate.cpp /catch_amalgamated.cpp /catch_amalgamated.hpp -I -std=c++17
```

The `[fine]` test cases finish normally. The `[infinite]` test case seems to run indefinitely, i.e. just keeps printing:
```shell
-------------------------------------------------------------------------------
Explicitly nested GENERATEs
Inside section 1
Inside section 2
-------------------------------------------------------------------------------
infinitegenerate.cpp:8
...............................................................................

infinitegenerate.cpp:14: FAILED:
CHECK( str1 == "myStr1.1" )
with expansion:
"myStr0.1" == "myStr1.1"
```

**Please note that I've indicated a line within this test case, which, if removed, the test case runs fine.**

**Platform information:**

- OS: **macOS 13.4.1**
- Compiler+version: **Clang v14.0.0**
- Catch version: **v3.4.0**

**Additional context**
It's worth noting that we haven't observed this issue when using the same code with **Catch2 v2.13.10**.

I've found the following old bug, which seems to be talking about a similar issue: https://github.com/catchorg/Catch2/issues/2025 Not sure it's related, though, as it was fixed quite a while ago.

Contributor guide

Open the contributing guide

Research direction

Start with the minimal reproduction in infinitegenerate.cpp, especially the explicitly nested GENERATE test and its reported line 14 failure, and run it against Catch2 v3.4.0. Compare its behavior with the working [fine] cases and Catch2 v2.13.10; done means nested GENERATEs terminate normally and the regression is covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.