[Performance]: dotnet/runtime tests are bottlenecked by MSBuild performance
- Dominant language
- C#
- Stars
- 5.5k
- Forks
- 1.5k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 141
Description
### Issue Description
The dotnet/runtime tests consist of thousands of tests in the `src/tests` tree. In the lab these tests take upwards of 20 minutes to build.
However, it looks like most of this time is in MSBuild. When I build the first fraction of these tests (1100 tests) locally, it takes MSBuild 6:50s.
If I run the same projects through Roslyn's "Replay" runner, which runs the csc commands from the binlog, this takes 1:39s.
That's a roughly 5x speedup.
The overall cost here is significant complexity in the runtime build. The other piece of important info is that the libraries tests, take only a few minutes to build. That means that if we were to build the entire test tree together we would delay starting the libraries tests by > 20 minutes. To counteract this we make the build substantially more complicated by splitting out a separate job in our pipeline just to build the runtime tests. If the overall build time were significantly reduced, we could remove a lot of complexity and delay in our CI testing.
### Steps to Reproduce
See dotnet/runtime src/tests tree. The command line I used to build was `src/tests/build.sh allTargets skipnative skipgeneratelayout skiptestwrappers checked x64 /p:LibrariesConfiguration=Release -bl`
### Data
MSBuild 6:50s.
Csc 1:39s.
### Analysis
_No response_
### Versions & Configurations
_No response_
### Regression
- [ ] yes
- [ ] no
### Regression Details
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.