benchmark-action / benchmark-action/github-action-benchmark
Running go benchmarks: Use `-count` or not?
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 184
- PR merge metrics
- No merged PRs in 30d
Description
We're setting up this action for our performance-critical go applications, and notice some variance in the results. In one situation it got so bad (or good) that PR branches now report performance regressions because the master branch by chance recorded a really good value.
I was wondering whether we can "smooth" that a bit by using `-count` when running the benchmarks, but am not sure how/if this action will handle that? If it "should just work", do you think it's a good idea?
Contributor guide
Research direction
No files or tests are named. Start by reproducing a Go benchmark run with and without `-count` through the GitHub Action, then inspect how the action invokes and processes benchmark results. Done means the action's support for repeated benchmark counts is established, with behavior documented or implemented and covered by regression testing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go, typescript
- Domain
- ci-cd, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100