Run all tests without arguments
- Dominant language
- C++
- Stars
- 10.4k
- Forks
- 1.8k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 8
Description
Hi! I find it confusing that the tests require arguments. I want to change it.
Here are my reasons:
1) Not all tests check for required arguments
https://github.com/google/benchmark/blob/7697796fe5a70edbc3394d0593ef82afeebb7ddf/test/spec_arg_test.cc#L63-L70 and so it's easy to run a test incorrectly.
2) There are GitHub issues mentioning that a test fails when the actual problem was that the correct arguments were not passed.
3) Arguments need to be duplicated between `test/CMakeLists.txt` and `test/BUILD` and they are actually not in sync now (`perf_counters_test` is run with different arguments).
My solution:
1) Create `test/default_arguments.h`:
```C++
// ...
void AddTestArguments(int &argc, char **&argv, std::initializer_list args={}) {
if (argc > 1) {
std::cout << "Warning: User is not expected to pass any command line arguments\n";
}
static std::vector new_argv;
new_argv.insert(new_argv.end(), argv, argv + argc);
new_argv.insert(new_argv.end(), args.begin(), args.end());
new_argv.push_back("--benchmark_min_time=0.01s");
argv = const_cast(new_argv.data());
argc = static_cast(new_argv.size());
}
```
2) Call it after each `main` function like this
```C++
int main(int argc, char* argv[]) {
AddTestArguments(argc, argv, {"--benchmark_counters_tabular=true"});
benchmark::MaybeReenterWithoutASLR(argc, argv);
RunOutputTests(argc, argv);
}
```
3) Cleanup the code
Sounds good?
Contributor guide
Assessment
This issue has not been assessed yet.