bencheeorg / bencheeorg/benchee
Ability to set a "reference" job
- Dominant language
- Elixir
- Stars
- 1.5k
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
Right now when benchmarks are compared, they are always compared to the fastest run. I think it would be useful in some situations to set a "reference" run and compare all the other runs to that.
One example would be the benchmark suite of `jason`, where I'd like to see all the other encoders reported relative to Jason itself, even though `jiffy` is the fastest most of the time. Right now, I have to do the additional computations manually, to get the desired numbers, that I can use to annoy people convincing them how fast Jason is 😉.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file or test is named. Trace the benchmark comparison path that selects the fastest run, then inspect how runs and encoders are represented; done means a chosen reference run can be used for all comparison results, including the Jason benchmark example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100