elm-explorations / elm-explorations/benchmark

Add warnings, errors, and tips to benchmark report

Open
#4 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Elm
Stars
28
Forks
4
PR merge metrics
No merged PRs in 30d

Description

**Issue by [jwoudenberg](https://github.com/jwoudenberg)**
_Monday Jun 05, 2017 at 12:35 GMT_
_Originally opened as https://github.com/BrianHicks/elm-benchmark/issues/13_

----

In the same vein as the elm compiler it wouldn't be really nice if elm-benchmark gave us warning, errors, and tips to help us write better benchmarks. From working with Brian a bit, I know he has tons of context on this, part of which could be automatically distributed in the benchmark report.

Below is an outline of some of Brian's tips I remeber, to give an idea of the type of helpful messages that could be displayed.

- **Standard deviations are large:**
- **Run counts are low**
- "Try to make the benchmark run faster by narrowing down the code it runs to the part you're actually trying to benchmark."
- "Perhaps you're not actually writing a micro-benchmark? Consider using a different tool. Here's a good one: ``."
- **Run counts are not low**
- "These programs are known to interfere with benchmark performance. Try closing them and running the benchmark again."
- "Did none if the above work? Please report an issue to help us make elm-benchmark better! ``"
- **Standard deviation is larger than delta:**
- "The benchmark is inconclusive."
- Should the delta even be shown in this scenario?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.