Question: Shared state file to improve RAR task performance
- Dominant language
- C#
- Stars
- 5.5k
- Forks
- 1.5k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 133
Description
This is not a performance issue rather it is a question on how to use the MS Build API to solve a performance issue I have.
### Issue Description
My build has 7000~ directories in a packages folder. The total time for RAR across the projects in the build is 30 minutes.
I currently inject many search paths via SearchPaths into the RAR task for compatibility reasons within my build.
I desire to have a faster build so optimizing RAR felt like a good place to start.
A solution I've thought about for a long time is to have a single state file for all of the projects as there is so much duplicated information that each RAR task discovers as it is a bit of a gold fish on a clean CI build.
I noticed in a recent drop of MSBuild that some new RAR properties had shown up with the words "precompute" and "statefile" which sounds very exciting.
A sample was constructed to have a single RAR instance walk the packages directory and hopefully crack open every single DLL in the folder and write that info the cache file which I can then provide to each project.
I've _assumed_ I am able to use **AssemblyInformationCacheOutputPath** and then feed that back into **AssemblyInformationCachePaths**.
```
public class PrecomputeReferenceCache : Task {
public string[] ReferencePath { get; set; }
public override bool Execute() {
System.Diagnostics.Debugger.Launch();
// 7000 items
var directories = Directory.GetDirectories(ReferencePath[0], "*", SearchOption.AllDirectories);
var sw = Stopwatch.StartNew();
ResolveAssemblyReference rar = new ResolveAssemblyReference() {
BuildEngine = this.BuildEngine,
SearchPaths = directories,
Assemblies = new ITaskItem[] {
new TaskItem("WebGrease, Version=1.6.5135.21930, Culture=neutral, PublicKeyToken=31bf3856ad364e35")
},
AssemblyInformationCacheOutputPath = @"C:\temp\rar.cache2",
StateFile = @"C:\temp\rar.cache",
};
rar.Execute();
sw.Stop();
var elapsed1 = sw.ElapsedMilliseconds;
sw = Stopwatch.StartNew();
ResolveAssemblyReference rar2 = new ResolveAssemblyReference()
{
BuildEngine = this.BuildEngine,
SearchPaths = directories,
Assemblies = new ITaskItem[] {
new TaskItem("WebGrease, Version=1.6.5135.21930, Culture=neutral, PublicKeyToken=31bf3856ad364e35")
},
AssemblyInformationCachePaths = new ITaskItem[]
{
new TaskItem(@"C:\temp\rar.cache2")
},
StateFile = @"C:\temp\rar.cache"
};
rar2.Execute();
sw.Stop();
var elapsed2 = sw.ElapsedMilliseconds;
return !Log.HasLoggedErrors;
}
```
elapsed1: 4528
elapsed2: 5773
The second RAR instance takes longer than the first instance and still grovels into each folder in SearchPaths.

The output files from the first RAR instance have some size to them

I was hoping that upToDateLocalFileStateCache would contain all of the metadata for each assembly so I can reuse this information for each project and avoid the repeated analysis cost.

**What I've learned**
It appears RAR will only store metadata for a DLL if it is present in the Assemblies list.
In my example I'm looking for WebGrease which only appears in a single folder of the 7000.
If I add 7000 unique entries to the Assemblies list will I get the behaviour I desire?
If I cannot get the precomputed state file to work I could look at removing as many SearchPaths as possible to speed up RAR.
### Versions & Configurations
```
Microsoft (R) Build Engine version 16.11.0+0538acc04 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
16.11.0.36601
```
### Regression?
No
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ResolveAssemblyReference and the AssemblyInformationCacheOutputPath, AssemblyInformationCachePaths, and StateFile properties shown in the sample task. Compare the first and second RAR invocations, including the continued SearchPaths traversal, to determine whether the cache is intended to cover this use case. Done means a documented, verified answer about sharing precomputed assembly state or a clearly scoped change request.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100