bencheeorg / bencheeorg/benchee_html

Compare Inputs in Results HTML

Open
#58 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Elixir
Stars
58
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Is there any way to configure `benchee` or `benchee_html` to compare the inputs on one graph?

I can do this with something like the following:

```
Benchee.run(
%{
"tx_benchmark 100" => fn -> TraditionalLedger.run_benchmark_file("./data/transactions_100.csv") end,
"tx_benchmark 1000" => fn -> TraditionalLedger.run_benchmark_file("./data/transactions_1000.csv") end,
"tx_benchmark 10000" => fn -> TraditionalLedger.run_benchmark_file("./data/transactions_10000.csv") end,
},
formatters: [
{Benchee.Formatters.Console, extended_statistics: true},
{Benchee.Formatters.HTML, file: "benchmarks/results.html"},
],
after_each: fn _input -> reset_database() end,
)
```

But doing it like the following "feels" better in terms of the code, but the HTML comparison charts will put these on separate pages:

```
Benchee.run(
%{
"tx_benchmark" => fn input -> TraditionalLedger.run_benchmark_file(input) end,
},
formatters: [
{Benchee.Formatters.Console, extended_statistics: true},
{Benchee.Formatters.HTML, file: "benchmarks/results.html"},
],
inputs: %{
"100" => "./data/transactions_100.csv",
"1_000" => "./data/transactions_1000.csv",
"10_000" => "./data/transactions_10000.csv",
},
after_each: fn _input -> reset_database() end,
)
```

And output the following:

Screen Shot 2021-12-04 at 11 25 59 AM

But I think this may just be a personal preference, but I could see there a use case for comparing `inputs` against each other in this fashion for benchmarking performance for different applications.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.