catchorg / catchorg/Catch2

Clock resolution estimation iterations limit

Open
#3,180 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Description**
Hello. I'm trying to run a benchmark on an embedded system. The system is `qemu-system-arm -machine mps2-an386` with 16MB RAM. It works, but I'm facing the following issue: during the `measure_environment` process, specifically in `estimate_clock_resolution`, it consumes all my available RAM and throws the `std::bad_alloc` exception.

The problem is that `estimate_clock_resolution` is only limited by time. At the same time, this is a hard-coded `clock_resolution_estimation_time` limit of 500ms and this cannot be configured without modifying the source code. It may be worth adding a limit on the number of iterations or allowing it to be configured. Alternatively, it may be possible to configure the time limit.

I suggest adding the ability to limit the number of iterations or the time limit using command-line parameters, similar to `--benchmark-warmup-time` e.g. `--benchmark-estimate-clock-time`.

**Additional context**
During the `measure_environment` process, the number of iterations reached 1'280'000, which resulted in the allocation of 1'280'000 `std::vector>` type elements in `std::vector> times(points)` and 1'280'001 `double` type elements in `std::vector deltas`.

Image

Contributor guide

Open the contributing guide

Research direction

Start by tracing measure_environment into estimate_clock_resolution and inspect how the 500ms clock_resolution_estimation_time controls collection of times and deltas. Compare the existing --benchmark-warmup-time option, then define and test a configurable time or iteration bound that prevents unbounded memory growth on the reported QEMU ARM setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.