catchorg / catchorg/Catch2

Integrate GENERATE with BENCHMARK

Open
#1,777 2 comments 15 reactions 0 assignees View on GitHub
Feature Request
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

It would be great if it would be possible parametrize benchmarks with generator expressions and get *one* table.

At the moment, multiple test cases are generated and multiple tables emitted...

For instance given the code:
```cpp
TEST_CASE("parametric_benchmark", "[!benchmark]") {
const size_t size = GENERATE(100, 1000);
CAPTURE(size);

auto data = Data(size);

BENCHMARK("algo0") {
return algo0(data);
};
BENCHMARK("algo1") {
return algo1(data);
};
}
```

the output is:
```
-------------------------------------------------------------------------------
parametric_benchmark
-------------------------------------------------------------------------------
../test.cpp:123
...............................................................................

benchmark name samples iterations mean
-------------------------------------------------------------------------------
algo0 100 1 1.27107 ms
algo1 100 1 57.789 us

-------------------------------------------------------------------------------
parametric_benchmark
-------------------------------------------------------------------------------
../test.cpp:123
...............................................................................

benchmark name samples iterations mean
-------------------------------------------------------------------------------
algo0 100 1 4.63871 ms
algo1 100 1 2.77958 ms
```

but much more readable would be:
```
-------------------------------------------------------------------------------
parametric_benchmark
-------------------------------------------------------------------------------
../test.cpp:123
...............................................................................

benchmark name samples iterations mean
-------------------------------------------------------------------------------
algo0 (size=100) 100 1 1.27107 ms
algo1 (size=100) 100 1 57.789 us
algo0 (size=1000) 100 1 4.63871 ms
algo1 (size=1000) 100 1 2.77958 ms
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.