Bug: AOT optimizer source generator output depends on hash order, so WinUI builds are not deterministic

Description

The AOT optimizer in WinRT.SourceGenerator produces different output for identical input depending on which compiler process runs it, so a WinUI 3 app built with <Deterministic>true</Deterministic> is not reproducible. Two clean builds of the same commit on the same machine give different assembly bytes, a different MVID and a different PDB whenever the compiler server (VBCSCompiler) is restarted in between.

The cause looks like the iteration order in GenerateVtableAttributes:

https://github.com/microsoft/CsWinRT/blob/2fc15e532febd0fe4b6bea10c9e5e3cde4fda172/src/Authoring/WinRT.SourceGenerator/AotOptimizer.cs#L960-L986

foreach (var vtableAttribute in vtableAttributes.ToImmutableHashSet())
{
    ...
    VtableEntry entry = new(vtableAttribute.Interfaces, vtableAttribute.GenericInterfaces, vtableAttribute.IsDelegate);
    bool vtableEntryExists = vtableEntryToVtableClassName.TryGetValue(entry, out var ccwClassName);
    if (!vtableEntryExists)
    {
        ccwClassName = GeneratorHelper.EscapeTypeNameForIdentifier(@namespace + vtableAttribute.ClassName);
        vtableEntryToVtableClassName.Add(entry, ccwClassName);
    }

ImmutableHashSet<VtableAttribute> enumerates in hash order. VtableAttribute's hash is built from strings, and string hash codes are randomized per process on .NET, so the order changes with every compiler process. That has two visible effects:

1. The shared vtable class is named after whichever type is visited first. In any XAML app, XamlTypeInfo.XamlUserType and XamlTypeInfo.XamlSystemBaseType have the same vtable entry, so the generated class is sometimes ..._XamlUserTypeWinRTTypeDetails and sometimes ..._XamlSystemBaseTypeWinRTTypeDetails. That changes the metadata string heap, the TypeDef table and every WinRTExposedType attribute that references it.
2. The order of the classes in WinRTCCWVtable.g.cs, and the order of the per-type *.WinRTVtable.g.cs sources, changes too. That changes the IL layout and the PDB document table even when the names happen to match.

The same code is in 2.2.0 (the generator shipped with Microsoft.Windows.SDK.NET.Ref 10.0.26100.57), in 2.3.1.260716.1 and on master. The other ToImmutableHashSet() loops in the same file (around L1787 and L1835 on master) feed generated output as well and look like they have the same problem.

Steps To Reproduce

A minimal WinUI 3 app is enough: an App.xaml and a MainWindow.xaml, with no user code beyond InitializeComponent().

Repro.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net10.0-windows10.0.26100.0</TargetFramework>
    <UseWinUI>true</UseWinUI>
    <Deterministic>true</Deterministic>
    <Platforms>x64</Platforms>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <WindowsPackageType>None</WindowsPackageType>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.WindowsAppSDK" Version="1.8.250907003" />
  </ItemGroup>
</Project>

App.xaml / App.xaml.cs:

<Application x:Class="Repro.App"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" />

namespace Repro;

public partial class App : Microsoft.UI.Xaml.Application
{
    public App() => InitializeComponent();

    protected override void OnLaunched(Microsoft.UI.Xaml.LaunchActivatedEventArgs args) => new MainWindow().Activate();
}

MainWindow.xaml / MainWindow.xaml.cs:

<Window x:Class="Repro.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <TextBlock Text="Hello" />
</Window>

namespace Repro;

public sealed partial class MainWindow : Microsoft.UI.Xaml.Window
{
    public MainWindow() => InitializeComponent();
}

Build it several times from clean, each in a fresh compiler server:

foreach ($i in 1..4) {
    dotnet build-server shutdown | Out-Null
    Remove-Item -Recurse -Force obj, bin -ErrorAction SilentlyContinue
    dotnet build -c Release -p:EmitCompilerGeneratedFiles=true | Out-Null
    $dll = Get-ChildItem bin -Recurse -Filter Repro.dll | Select-Object -First 1
    $gen = Get-ChildItem obj -Recurse -Filter WinRTCCWVtable.g.cs | Select-Object -First 1
    $names = (Select-String $gen.FullName 'class (\w+)WinRTTypeDetails' | ForEach-Object { $_.Matches[0].Groups[1].Value }) -join ','
    "$i $((Get-FileHash $dll.FullName).Hash.Substring(0, 16)) $names"
}

Output (four builds, four different assemblies):

1 360406F04E29D578 Repro_Repro_XamlTypeInfo_XamlSystemBaseType,Repro_App,Repro_MainWindow,Repro_Repro_XamlTypeInfo_XamlMetaDataProvider,Repro_Repro_XamlTypeInfo_XamlMember
2 045B58BA2F275AB3 Repro_App,Repro_Repro_XamlTypeInfo_XamlMember,Repro_Repro_XamlTypeInfo_XamlMetaDataProvider,Repro_MainWindow,Repro_Repro_XamlTypeInfo_XamlUserType
3 8E89E5456986BCE1 Repro_App,Repro_Repro_XamlTypeInfo_XamlUserType,Repro_Repro_XamlTypeInfo_XamlMember,Repro_MainWindow,Repro_Repro_XamlTypeInfo_XamlMetaDataProvider
4 E5A451EC2C946CCA Repro_Repro_XamlTypeInfo_XamlMember,Repro_Repro_XamlTypeInfo_XamlSystemBaseType,Repro_Repro_XamlTypeInfo_XamlMetaDataProvider,Repro_App,Repro_MainWindow

If the compiler server is reused between builds, the hashes match, because the string hash seed is the same. That is why this is easy to miss locally and shows up in CI, where every job gets a fresh process.

With <CsWinRTAotOptimizerEnabled>false</CsWinRTAotOptimizerEnabled> the four builds come out identical.

Expected Behavior

Identical input produces identical generator output, and so an identical assembly, regardless of which compiler process runs the generator. That is what <Deterministic>true</Deterministic> and reproducible-build tooling rely on.

A fix that keeps the current deduplication is to iterate in a stable order, e.g. vtableAttributes.Distinct().OrderBy(static v => v.Namespace, StringComparer.Ordinal).ThenBy(static v => v.ClassName, StringComparer.Ordinal) (plus whatever else makes an entry unique), in GenerateVtableAttributes and the other ToImmutableHashSet() loops whose output is emitted.

Version Info

• WinRT.SourceGenerator.dll 2.2.0.48161, from Microsoft.Windows.SDK.NET.Ref 10.0.26100.57 (the default for net10.0-windows10.0.26100.0). The code is unchanged in CsWinRT 2.3.1.260716.1 and on master at 2fc15e5.
• .NET SDK 10.0.303
• Microsoft.WindowsAppSDK 1.8.250907003
• Windows 11 (x64). Also seen between GitHub Actions windows-latest (x64) and windows-11-arm runners.

Additional Context

We found this through a build gate that requires managed assemblies to be byte-identical across two builds. Our workaround is CsWinRTAotOptimizerEnabled=false, which is fine for an app that is neither trimmed nor Native AOT, but it isn't an option for apps that need the optimizer.

Start in src/Authoring/WinRT.SourceGenerator/AotOptimizer.cs at GenerateVtableAttributes around lines 960-986, then inspect the other ToImmutableHashSet() loops around lines 1787 and 1835. Use the supplied minimal WinUI project and fresh compiler-server build loop to check generated sources and assembly hashes. Done means identical generated output and assembly bytes across clean builds with different compiler processes.