`--benchmark-samples` does not seem to work as expected
- Dominant language
- C++
- Stars
- 21.5k
- Forks
- 3.5k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 2
Description
Hi all,
I have some functions I'm trying to benchmark, and some of them can only be called once (internal mechanisms prevent a user to call them twice). The benchmarked function **cannot** be modified for obvious reasons, and there is no way I can prevent them from checking they are called only once.
After few search, I found the `--benchmark-samples` option, which allows to set the number of times the code within `BENCHMARK` macro is called. Unfortunately it seems it does not act as intended. I even used all the arsenal of available options to control benchmarks as follows `--benchmark-samples 1 --benchmark-resamples 1 --benchmark-warmup-time 0 --benchmark-no-analysis`
Nothing changed.
What am I doing wrong? What should I do instead ?
Here is a sample code that produces the issue.
```C++
static uint8_t calls = 0;
static void function_callable_only_once() {
if (calls > 0) throw std::exception("Only one call is allowed.");
calls += 1;
}
BENCHMARK("generate_setup_commitments") {
std::cout << "********** THIS IS SPARTA *************" << std::endl;
REQUIRE_NOTHROW(function_callable_only_once());
};
```
The output is the following :
```bash
benchmark name samples iterations mean
-------------------------------------------------------------------------------
generate_setup_commitments ********** THIS IS SPARTA *************
1 1 ********** THIS IS SPARTA *************
stupid_test.cpp:25: FAILED:
REQUIRE_NOTHROW(function_callable_only_once())
due to unexpected exception with message:
Only one call is allowed.
```
Where you can see that the string `********** THIS IS SPARTA *************` is printed more than once.
Please help/advise.
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.