dotnet / dotnet/BenchmarkDotNet
Summary Save/Load/Combine
- Dominant language
- C#
- Stars
- 11.5k
- Forks
- 1.1k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 11
Description
Now BenchmarkDotNet allows to create a nice set of jobs for different environments. However, sometimes it's impossible to get all desired result with a single run. Examples:
* Comparing performance before and after refactoring
* Comparing performance between different version of a NuGet package (probably will be implemented in the future, but it's not available for now)
* Comparing performance between different OS (Windows/Linux/MacOS)
So, I suggest to add `Save`/`Load`/`Combine` methods which will help to solve this problems. An example:
```cs
// Part 1: WIndows
var summary = BenchmarkRunner.Run();
summary.Save("windows.json");
// Part 2: Linux
var summary = BenchmarkRunner.Run();
summary.Save("linux.json")
// Part 3: Common summary table
var summaryWindows = Summary.Load("windows.json");
var summaryLinux = Summary.Load("linux.json");
var summary = Summary.Combine(summaryWindows, summaryLinux);
```
Some additional thoughts / points for discussions.
* Is it ok to save summary in the json format?
* Probably it would be nice to have an API for adding additional columns to the current summary like `var newSummary = mySummary.WithAdditionalColumn("VersionOfMyLibrary", "0.10.0")`.
@adamsitnik, @mattwarren, @ig-sinicyn, @terrajobst, what do you think?
Contributor guide
Assessment
This issue has not been assessed yet.