google / google/benchmark

[BUG] With NDEBUG, early-returning without SkipWithError leads to indefinite retries

Open
#1,754 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
10.4k
Forks
1.8k
Avg merge
2d 4h
Merged PRs (30d)
8

Description

**Describe the bug**
With NDEBUG, early-returning without SkipWithError leads to indefinite retries

**System**
Which OS, compiler, and compiler version are you using:
- OS: ```6.5.13-1rodete2-amd64 (building for Android cuttlefish)```
- Compiler and version: clang --version: ```Android (11368308, +pgo, +bolt, +lto, +mlgo, based on r510928) clang version 18.0.0 (https://android.googlesource.com/toolchain/llvm-project 477610d4d0d988e69dbc3fae4fe86bff3f07f2b5)```

**To reproduce**
Steps to reproduce the behavior:
1. Run https://cs.android.com/android/_/android/platform/hardware/interfaces/+/f850de6732287b3c5539a56751d21a4a1b170f42:vibrator/bench/benchmark.cpp;bpv=1;bpt=0;drc=e29acd8194c6df2fc81613cebdb2dc4507ec4c1b
2. Benchmark prints the context, then hangs

Note how the benchmark functions early return before reaching the measure loop. Also note that Android compiles with NDEBUG set.

Minimal repro case (**compile with `-DNDEBUG=1`**)
```
static void BM_Foobar(benchmark::State& state) {
return;
}
// Register the function as a benchmark
BENCHMARK(BM_Foobar);
```

**Expected behavior**

Prints an error & skips / aborts, instead of retrying indefinitely.

**Root cause**

https://github.com/google/benchmark/blob/b04cec1bf90c3d8e47739bb3271607a18d8b5106/src/benchmark_runner.cc#L133 does not trigger an abort if NDEBUG is set; instead, the benchmark function gets retried indefinitely.

Verbose logs:
```c++
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
-- LOG(2): Ran in 0/0
-- LOG(2): Running VibratorBench_V1_0/on for 1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.