catchorg / catchorg/Catch2

[Query] Best practices for integration testing?

Open
#1,747 0 comments 9 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

Right now I use Catch for unit testing: If I have a class `Foo`, I write a test to verify the interface and observable behavior of class `Foo`. I mock out dependencies that `Foo` has in order to keep unit tests deterministic and free from outside interference.

The next phase of testing, integration testing, would allow me to make sure class `Foo` and `Bar` work well together. In Catch terms, this could also be a test case. It would be a really big one, because you'd have to do a lot of application "set-up" stuff, to initialize subsystems and prepare for the actual high-level behavior to be run. You'd have to potentially mock network services, databases, files, and other things.

Assuming that Catch is suitable for this, in what way would you structure your CI process to execute these two tiers of tests? One way I can think of is to come up with some global tags. For example, `[UnitTest]` and `[IntegrationTest]`. Then my CI script might do this:

```
#!/usr/bin/env bash
set -e # Script should stop if any test step fails

# Execute unit tests
MyTests.exe [UnitTest]

# Execute integration tests, only if the above didn't fail
MyTests.exe [IntegrationTest]
```

This would allow your lower-level tests to execute prior to your high-level tests. The justification here is that you don't want to waste time running integration tests if the foundations on which they are built are also dysfunctional.

The more simple, possibly "faster" option would be to just do `MyTests.exe`, with no tag specified, to run everything together.

Has anyone set up a continuous integration pipeline with these two tiers of tests? How have you structured it? What do the catch developers themselves recommend?

I realize this is a bit off-topic, as it's more focused on best practices with Catch involved, but I can't think of a better place to ask. Thanks in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.