dotnet / dotnet/msbuild

BenchmarkDotNet msbuild task benchmarks

Open
#5,746 3 comments 2 reactions 0 assignees View on GitHub
needs-design triaged
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 8h
Merged PRs (30d)
141

Description

Hi there,

I am creating this issue to track the ideas and progress of adding BenchmarkDotNet-based benchmarks to msbuild.

The idea is to be able to benchmark MSBuild tasks independently and help catch regressions and improve performance.

I started experimenting with this here https://github.com/mfkl/msbuild/tree/benchmark. Here's a basic example:

```
[GlobalSetup(Target = nameof(ConflictBetweenBackAndForeVersionsNotCopyLocal))]
public void ConflictBetweenBackAndForeVersionsNotCopyLocalSetup()
{
t = new ResolveAssemblyReference
{
Assemblies = new ITaskItem[]
{
new TaskItem("D, Version=2.0.0.0, Culture=neutral, PublicKeyToken=aaaaaaaaaaaaaaaa"),
new TaskItem("D, Version=1.0.0.0, Culture=neutral, PublicKeyToken=aaaaaaaaaaaaaaaa")
},
BuildEngine = new MockEngine(),
SearchPaths = new string[] {
s_myLibrariesRootPath, s_myLibraries_V2Path, s_myLibraries_V1Path
}
};
}

[Benchmark]
public void ConflictBetweenBackAndForeVersionsNotCopyLocal()
{
t.Execute();
}
```

I've been looking at existing unit tests as a base to start this work. I think that time consuming tasks, like RAR (even more so with the current perf-focused RAR refactoring), are good candidate for writing benchmarks.

If you have any opinion about this, your thoughts are welcome.

Regarding the infra, I'm not familiar with helix/arcade but it seems to be favored for perf tests in other dotnet projects. Feel free to let me know what would be the plan regarding this, I'm fine working on this locally for now though.

Benchmarks can be ran with `dotnet run -c Release`

Relevant:
https://github.com/xamarin/xamarin-android/blob/master/tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs
https://github.com/xamarin/xamarin-android/blob/master/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.