jenkinsci / jenkinsci/benchmark-plugin
Threshold specification does not allow positive improvements in benchmarks
Open
enhancement
- Dominant language
- Java
- Stars
- 7
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
### What feature do you want to see added?
The current threshold classes all take absolute values of changes (e.g. absolute delta in execution time compared to last build, or absolute percentage of change in execution time compared to last build), but in all cases positive changes (e.g. faster execution) will still be regarded as a negative outcome and will mark the benchmark as failed. It should be possible to mark benchmarks as passing when the execution time has improved or is equal to the previous build, and only fail those where execution time is actually worse.
### Upstream changes
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.