google / google/benchmark

[BUG] Minimum benchmark time with multiple threads

Open
#2,117 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
10.4k
Forks
1.8k
Avg merge
2d 4h
Merged PRs (30d)
8

Description

**Describe the bug**
https://github.com/google/benchmark/pull/1836 fixed https://github.com/google/benchmark/issues/1834 and now the reported output makes more sense. However, the minimum benchmark time doesn't seem to be respected.

**System**
Arch Linux, GCC 15.2.1, `benchmark` 1.9.5-1

**To reproduce**
Consider
```c++
#include

static void BM_MultiThreaded(benchmark::State& state) {
int a = 0;

for (auto _ : state) {
for (int i = 0; i < 1000; i++) {
benchmark::DoNotOptimize(a++);
}
}
}
BENCHMARK(BM_MultiThreaded)->ThreadRange(1, 8);

BENCHMARK_MAIN();
```

The output is somewhat expected:
```
---------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------
BM_MultiThreaded/threads:1 275 ns 274 ns 2585850
BM_MultiThreaded/threads:2 273 ns 272 ns 2259184
BM_MultiThreaded/threads:4 279 ns 278 ns 2449204
BM_MultiThreaded/threads:8 308 ns 307 ns 2148808
```
However, notice that the number of iterations stays constant - it should increase as more threads are used to run the iterations. For example:
```
$ /usr/bin/time -v ./benchmark-multithreaded --benchmark_min_time=1s --benchmark_filter=threads:8
[...]
User time (seconds): 1.72
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.25
```

With previous versions of `benchmark`, this could be fixed with `UseRealTime()` but that doesn't seem to work anymore since (probably) https://github.com/google/benchmark/pull/1836.

**Expected behavior**
Each benchmark should spend the minimum time (per thread).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.