catchorg / catchorg/Catch2

Allow Tags ([!mayfail]) in SECTION/WHEN/THEN to support TDD in existing test suites

Open
#1,385 1 comment 10 reactions 0 assignees View on GitHub
Feature Request
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

## Description
I am running a BDD-style component test consisting of a

```cpp
SCENARIO("large moduletest with some complicated testing harness","[!mayfail]"){
init_common_test_harness();
GIVEN("more initcode"){
WHEN("specific action on harness"){
THEN(){}
}}
... (lots of nested GIVEN/WHEN/THEN blocks)
}
```

I would like to use Catch as a living spec in a test-driven design approach and write testing code before/while implementing the actual functionality.
In order to not fail the full test suite and upset colleagues, I am adding [!mayfail] to the scenario each time I add a new WHEN/THEN test.
Since this marks all sections of the scenario as mayfail, edits by colleagues that break existing functionality don't surface while I'm working on the new section (since Catch returns success regardless of errors in the full scenario).

I would like to be able to add tags, esp. mayfail to individual sections, to mark them as "not yet working" without disabling my full testsuite, as in

```cpp
SCENARIO("large moduletest with some complicated testing harness"){
init_common_test_harness();
GIVEN("more initcode"){
WHEN("specific action on harness"){
THEN(){}
}}
... // << Failures until here lead to nonzero exit status
WHEN("new action on harness"){
THEN("succeed","[!mayfail]"){} // Failures in this section are ignored
}}
```

This is not currently possible since GIVEN/WHEN/THEN/SECTION macros don't support tags, only SCENARIO/TEST_CASE does.

Splitting my tests into multiple scenarios would violate the DRY principle, since all sections depend on a a large common test harness, and exchanging GIVEN/WHEN with SCENARIO would contradict the BDD style and make the tests harder to read/understand.

### Extra information
* Catch version: v2.3.0
* Operating System: Win 10
* Compiler+version: msvc19

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.