dotnet / dotnet/BenchmarkDotNet

Summary Save/Load/Combine

Open
#305 13 comments 6 reactions 1 assignee Claimed by @AndreyAkinshin View on GitHub
Area:Exporters enhancement
Dominant language
C#
Stars
11.5k
Forks
1.1k
Avg merge
4d 11h
Merged PRs (30d)
11

Description

Now BenchmarkDotNet allows to create a nice set of jobs for different environments. However, sometimes it's impossible to get all desired result with a single run. Examples:
* Comparing performance before and after refactoring
* Comparing performance between different version of a NuGet package (probably will be implemented in the future, but it's not available for now)
* Comparing performance between different OS (Windows/Linux/MacOS)

So, I suggest to add `Save`/`Load`/`Combine` methods which will help to solve this problems. An example:
```cs
// Part 1: WIndows
var summary = BenchmarkRunner.Run();
summary.Save("windows.json");
// Part 2: Linux
var summary = BenchmarkRunner.Run();
summary.Save("linux.json")
// Part 3: Common summary table
var summaryWindows = Summary.Load("windows.json");
var summaryLinux = Summary.Load("linux.json");
var summary = Summary.Combine(summaryWindows, summaryLinux);
```

Some additional thoughts / points for discussions.
* Is it ok to save summary in the json format?
* Probably it would be nice to have an API for adding additional columns to the current summary like `var newSummary = mySummary.WithAdditionalColumn("VersionOfMyLibrary", "0.10.0")`.

@adamsitnik, @mattwarren, @ig-sinicyn, @terrajobst, what do you think?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.