benchmark-action / benchmark-action/github-action-benchmark
Allow choice of benchmark stat used for `pytest`
Open
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 184
- PR merge metrics
- No merged PRs in 30d
Description
`pytest-benchmark` automatically computes a fairly comprehensive set of stats across multiple runs of each benchmark, including minimum, mean, median, etc. runtime, and Operations Per Second (calculated as the inverse of the mean runtime).
It's commonly recommended that benchmark comparisons use the minimum runtime, as it is more stable than other statistics. However, the actions are currently hardcoded to use the `ops` statistic.
Is it possible to make this configurable?
(This request may apply more widely than `pytest`, but I'm not familiar with the other output formats that are handled)
Contributor guide
Assessment
This issue has not been assessed yet.