dotnet / dotnet/BenchmarkDotNet

Smart way to detect benchmarks without a steady state

Open
#594 0 comments 1 reaction 1 assignee Claimed by @morgan-kn View on GitHub
Area:Engine enhancement
Dominant language
C#
Stars
11.5k
Forks
1.1k
Avg merge
6d 11h
Merged PRs (30d)
12

Description

Sometimes, benchmarks don't have a steady state. For example, each benchmark iteration can take more time than the previous one. In this case, our default run strategy will run new iterations again and again (because it tries to find a steady state). We should have a smart way to detect such kind of situation, terminate the process, and warn users about it. We should also beware of false alarms: in some cases, it's a legal situation (the main logic of a benchmark can require dozens of warmup iteration before it reaches the steady state).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.