Custom CulmulativeReporterBase receives odd children order when dealing with Skipped tests
- Dominant language
- C++
- Stars
- 21.5k
- Forks
- 3.5k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 2
Description
**Describe the bug**
I'm building a reporter from CulmulativeReporterBase. I'm iterating the children of the m_testRun, and walking through each child SectionNode.
```
void LogSectionNode(const Catch::CumulativeReporterBase::SectionNode& node, const std::string& prefix)
{
std::cout << prefix << "Section: " << node.stats.sectionInfo.name <<
" (child count: " << node.childSections.size() << " passed:" << node.stats.assertions.passed << " skipped:" << node.stats.assertions.skipped << ")\n";
for(const auto& childSection : node.childSections)
{
LogSectionNode(*childSection, prefix + " ");
}
}
void ReproReporter::testRunEndedCumulative()
{
for (const auto& childNode : m_testRun->children)
{
std::cout << "test node: " << childNode->value.testInfo->name << '\n';
for (const auto& childSection : childNode->children)
{
LogSectionNode(*childSection, " ");
}
}
}
```
I'm running it against a Scenario such as this:
```
SCENARIO("Repro")
{
GIVEN("ReproGiven")
{
WHEN("ReproWhenA")
{
THEN("ReproThenA1")
{
SUCCEED();
}
THEN("ReproThenA2")
{
SUCCEED();
}
}
WHEN("ReproWhenB")
{
//SKIP("skip");
SUCCEED();
}
}
}
```
When I uncomment the skip, I get very odd output:
```
test node: Scenario: Repro
Section: Scenario: Repro (child count: 2 passed:0 skipped:0)
Section: When: ReproWhenB (child count: 2 passed:0 skipped:1)
Section: When: ReproWhenA (child count: 2 passed:1 skipped:0)
Section: Then: ReproThenA1 (child count: 0 passed:1 skipped:0)
Section: Then: ReproThenA2 (child count: 0 passed:1 skipped:0)
Section: Given: ReproGiven (child count: 0 passed:0 skipped:1)
Section: Given: ReproGiven (child count: 0 passed:0 skipped:0)
```
Note that the scenario has two children, once called ReproWhenB and another called ReproGiven. There's a second ReproGIven under ReproWhenB, that contains the actual skipped test.
**Expected behavior**
Compare that to the output when I don't skip:
```
test node: Scenario: Repro
Section: Scenario: Repro (child count: 1 passed:1 skipped:0)
Section: Given: ReproGiven (child count: 2 passed:1 skipped:0)
Section: When: ReproWhenA (child count: 2 passed:1 skipped:0)
Section: Then: ReproThenA1 (child count: 0 passed:1 skipped:0)
Section: Then: ReproThenA2 (child count: 0 passed:1 skipped:0)
Section: When: ReproWhenB (child count: 0 passed:1 skipped:0)
```
Now the scenario has one child, ReproGIven, which has two children, WhenA and WhenB. Nothing here looks weird.
**Reproduction steps**
https://github.com/tenpn/catch2-skipreporter-repro
download, build with `xbuild build`, comment/uncomment the skip in src/ReproScenario.cpp, run the tests with `xbuild run binary -r repro`.
**Platform information:**
- OS: **Windows 10**
- Compiler+version: **VS 2022 Community**
- Catch version: **v3.7.0**
Contributor guide
Assessment
This issue has not been assessed yet.