dotnet / dotnet/msbuild

[Performance]: dotnet/runtime tests are bottlenecked by MSBuild performance

Open
#10,021 11 comments 0 reactions 1 assignee Assigned to @MichalPavlik View on GitHub
Area: Performance Priority:2 triaged
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 8h
Merged PRs (30d)
141

Description

### Issue Description

The dotnet/runtime tests consist of thousands of tests in the `src/tests` tree. In the lab these tests take upwards of 20 minutes to build.

However, it looks like most of this time is in MSBuild. When I build the first fraction of these tests (1100 tests) locally, it takes MSBuild 6:50s.

If I run the same projects through Roslyn's "Replay" runner, which runs the csc commands from the binlog, this takes 1:39s.

That's a roughly 5x speedup.

The overall cost here is significant complexity in the runtime build. The other piece of important info is that the libraries tests, take only a few minutes to build. That means that if we were to build the entire test tree together we would delay starting the libraries tests by > 20 minutes. To counteract this we make the build substantially more complicated by splitting out a separate job in our pipeline just to build the runtime tests. If the overall build time were significantly reduced, we could remove a lot of complexity and delay in our CI testing.

### Steps to Reproduce

See dotnet/runtime src/tests tree. The command line I used to build was `src/tests/build.sh allTargets skipnative skipgeneratelayout skiptestwrappers checked x64 /p:LibrariesConfiguration=Release -bl`

### Data

MSBuild 6:50s.
Csc 1:39s.

### Analysis

_No response_

### Versions & Configurations

_No response_

### Regression

- [ ] yes
- [ ] no

### Regression Details

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.