benchmark-action / benchmark-action/github-action-benchmark
Concerns about reiability of GA
Open
question
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 184
- PR merge metrics
- No merged PRs in 30d
Description
We've been [observing](https://github.com/clap-rs/clap/pull/1755) large fluctuation in benchmark results on essentially empty commits. It's apparently because GitHub Actions runs benches on different machines (come to think of it, it's sensible) with different load score.
It doesn't look like GA should be used for benchmarking to me.
Contributor guide
Assessment
This issue has not been assessed yet.