benchmark-action / benchmark-action/github-action-benchmark
Feature request: Support iai Rust benchmarks
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 184
- PR merge metrics
- No merged PRs in 30d
Description
Rust benchmarking can be noisy in virtualized environments (GH Actions) and the best way to run the benchmarks in those is probably [iai](https://github.com/bheisler/iai). Its format seems to be unsupported my workflow runs the benchmarks and prints them out to file : https://github.com/kirillbobyrev/pabi/runs/5079239022
From the [docs](https://bheisler.github.io/criterion.rs/book/iai/iai.html):
> It is intended as a complement to Criterion.rs; among other things, it's useful for reliable benchmarking in CI.
And
> For benchmarks that run in CI (especially if you're checking for performance regressions in pull requests on cloud CI) you should use Iai. For benchmarking on Windows or other platforms that Valgrind doesn't support, you should use Criterion-rs. For other cases, I would advise using both. Iai gives more precision and scales better to larger benchmarks, while Criterion-rs allows for excluding setup time and gives you more information about the actual time your code takes and how strongly that is affected by non-determinism like threading or hash-table randomization. If you absolutely need to pick one or the other though, Iai is probably the one to go with.
Contributor guide
Assessment
This issue has not been assessed yet.