Use of PauseTiming() and ResumeTiming() gives completely different results
- Dominant language
- C++
- Stars
- 10.4k
- Forks
- 1.8k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 8
Description
```cpp
#include
#include
#include "trie.hpp"
template
void trie_find(benchmark::State& state, ExtraArgs&&... extra_args) {
std::string filename(std::forward(extra_args)...);
std::basic_ifstream file(filename);
Trie trie;
std::basic_string word;
while (std::getline(file, word)) {
trie.insert(word);
}
file.close();
for (auto _ : state) {
// state.PauseTiming();
file.open(filename);
std::basic_string word;
while (std::getline(file, word)) {
// state.ResumeTiming();
trie.find(word);
// state.PauseTiming();
}
}
}
BENCHMARK_CAPTURE(trie_find, lulzsec, "wordlists/lulzsec.txt");
BENCHMARK_CAPTURE(trie_find, cain, "wordlists/cain.txt");
BENCHMARK_CAPTURE(trie_find, hak5, "wordlists/hak5.txt");
BENCHMARK_CAPTURE(trie_find, john, "wordlists/john.txt");
BENCHMARK_CAPTURE(trie_find, bible, "wordlists/bible.txt");
BENCHMARK_MAIN();
```
```bash
g++ benchmark.cpp -lbenchmark -O3
```
Without `PauseTiming()` and `ResumeTiming()`
```bash
2019-07-25 04:44:02
Running ./a.out
Run on (4 X 2700 MHz CPU s)
CPU Caches:
L1 Data 32K (x2)
L1 Instruction 32K (x2)
L2 Unified 256K (x2)
L3 Unified 3072K (x1)
Load Average: 1.59, 0.76, 0.53
------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------
trie_find/lulzsec 578711892 ns 576002624 ns 1
trie_find/cain 47.3 ns 47.2 ns 13669606
trie_find/hak5 37.7 ns 37.6 ns 18620521
trie_find/john 37.7 ns 37.7 ns 18679699
trie_find/bible 24.6 ns 24.5 ns 28378274
```
With `PauseTiming()` and `ResumeTiming()`
```bash
2019-07-25 04:45:41
Running ./a.out
Run on (4 X 2700 MHz CPU s)
CPU Caches:
L1 Data 32K (x2)
L1 Instruction 32K (x2)
L2 Unified 256K (x2)
L3 Unified 3072K (x1)
Load Average: 0.49, 0.64, 0.51
------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------
trie_find/lulzsec 765193458 ns 764815007 ns 1
trie_find/cain 818307 ns 818694 ns 629
trie_find/hak5 698936 ns 698946 ns 1769
trie_find/john 701939 ns 697227 ns 1772
trie_find/bible 679012 ns 676065 ns 1701
```
I have no idea why is that happening. Can you please help me?
Contributor guide
Assessment
This issue has not been assessed yet.