catchorg / catchorg/Catch2

Info collection is incapacitatingly slow

Open
#2,601 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Describe the bug**
When running many tests, then even when the test code itself is minimal and fast, info collection for Catch2's test output seems to be taking way too long, making it effectively unusable.

**Expected behavior**
Test should finish way more quickly, probably by skipping the collection of so much successful-assertion data, at least when `--success` is not specified.

**Reproduction steps**

```c++
TEST_CASE("test scenario", "[bla_bla]") {
for (auto i = 1; i < 10'000'000; i++) {
CHECK(i >= 1);
}
}
```

This code (incrementing a counter to 10 million) runs on my machine in 25ms without Catch2, but takes about 10 seconds under Catch2 => a 400x slowdown.

**Platform information:**

- OS: **Ubuntu 20.04**
- Compiler+version: **GCC v9.4.0**
- Catch version: **v3.1.1**

**Additional context**
When testing templated data structures and algorithms, it's often useful and important to test many, many different types and values in different usage patterns, to discover edge-cases. The current long run times make this a practical impossibility.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.