jenkinsci / jenkinsci/benchmark-plugin

Threshold specification does not allow positive improvements in benchmarks

Open
#47 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
7
Forks
4
PR merge metrics
No merged PRs in 30d

Description

### What feature do you want to see added?

The current threshold classes all take absolute values of changes (e.g. absolute delta in execution time compared to last build, or absolute percentage of change in execution time compared to last build), but in all cases positive changes (e.g. faster execution) will still be regarded as a negative outcome and will mark the benchmark as failed. It should be possible to mark benchmarks as passing when the execution time has improved or is equal to the previous build, and only fail those where execution time is actually worse.

### Upstream changes

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.