elm-explorations / elm-explorations/benchmark
Add warnings, errors, and tips to benchmark report
- Dominant language
- Elm
- Stars
- 28
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
**Issue by [jwoudenberg](https://github.com/jwoudenberg)**
_Monday Jun 05, 2017 at 12:35 GMT_
_Originally opened as https://github.com/BrianHicks/elm-benchmark/issues/13_
----
In the same vein as the elm compiler it wouldn't be really nice if elm-benchmark gave us warning, errors, and tips to help us write better benchmarks. From working with Brian a bit, I know he has tons of context on this, part of which could be automatically distributed in the benchmark report.
Below is an outline of some of Brian's tips I remeber, to give an idea of the type of helpful messages that could be displayed.
- **Standard deviations are large:**
- **Run counts are low**
- "Try to make the benchmark run faster by narrowing down the code it runs to the part you're actually trying to benchmark."
- "Perhaps you're not actually writing a micro-benchmark? Consider using a different tool. Here's a good one: ``."
- **Run counts are not low**
- "These programs are known to interfere with benchmark performance. Try closing them and running the benchmark again."
- "Did none if the above work? Please report an issue to help us make elm-benchmark better! ``"
- **Standard deviation is larger than delta:**
- "The benchmark is inconclusive."
- Should the delta even be shown in this scenario?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.