Support incremental NativeAOT compilation
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Description
NativeAOT should support incremental compilation on every supported platform and architecture. Once the implementation reaches the required correctness and compatibility bar, incremental compilation should be the default behavior for local NativeAOT builds, with an option to force a clean build when needed.
A full NativeAOT build is not merely a final packaging step after managed development. Developers need to iterate on true NativeAOT outputs because they can differ significantly from managed builds in:
- Executable and deployment size
- Trimming and reflection behavior
- Generic sharing and code generation
- Static initialization
- Native interop and exported entry points
- Startup time, throughput, and memory use
- Platform-specific runtime behavior
Requiring a full whole-program compilation after every change makes it impractical to inspect these characteristics during normal development, especially for large applications.
### Desired experience
1. The first `dotnet publish -p:PublishAot=true` may perform a clean whole-program compilation.
2. Subsequent builds automatically reuse all compilation work that remains valid.
3. Incremental compilation is enabled by default rather than requiring experimental environment variables or direct ILC invocation.
4. The result has the same whole-program, single-file compilation shape as a clean production NativeAOT build, so its size, performance, and behavior are representative of the final application.
5. Invalidation is automatic and fail-safe. If reuse cannot be proven correct, the build performs a clean compilation.
6. Build output explains whether incremental compilation was used, what was invalidated, and why a clean fallback occurred.
7. Developers can explicitly request a clean build or disable incremental state for diagnostics.
8. The feature works across all NativeAOT-supported platforms. Windows, Linux, and macOS on x64 and ARM64 are the minimum expected desktop coverage.
### Representative data
A representative very large production application produced:
| Metric | Measurement |
| --- | ---: |
| Complete clean publish | 951.9 s (15m 51.9s) |
| Subsequent unchanged publish | approximately 784.0 s (13m 4s) |
| Matched clean ILC compilation | 640.395 s (10m 40.4s) |
| Native link | 36.55 s |
| Native object size | 3,744,339,247 bytes |
| Native executable size | approximately 1.039 GB |
| Complete publish output | approximately 1.455 GB |
| Dependency-graph nodes | 30,666,594 |
| Peak ILC working set | 35.16 GB |
| Cumulative managed allocation | 123.80 GB |
A narrowly gated feasibility prototype retained valid compiler state for one small method-body change:
| Step | Clean path | Incremental prototype | Impact |
| --- | ---: | ---: | ---: |
| ILC object generation/update | 640.395 s | 291.837 ms | 2,194.36x faster |
| Native link | 36.55 s | 36.55 s | unchanged |
| ILC + link developer loop | 676.945 s | 36.842 s | 18.37x faster |
| Time saved per eligible iteration | — | 640.103 s | 10m 40.1s saved |
The incremental object exactly matched an independent clean object by SHA-256. The prototype reused 13,455,307 of 13,455,308 object nodes and patched one byte.
These measurements demonstrate the amount of reusable work available in a large NativeAOT build. They are not a general performance promise: the prototype first paid for a clean compilation, retained approximately 34–36 GB of state, and accepted only a very narrow class of edits. A comparable complete incremental `dotnet publish` was not measured.
### Output-shape requirement
Incremental builds must remain representative of the final clean NativeAOT output. An incremental mode that materially changes whole-program analysis, code generation, generic handling, metadata, layout, or optimization decisions is insufficient for workflows that inspect final application size, performance, and behavior.
Where deterministic output is expected, incremental and clean builds should ideally be byte-identical. At minimum, they must be semantically equivalent and produce the same production compilation model and optimization opportunities.
### Relationship to `IlcMultiModule`
The existing unsupported `IlcMultiModule=true` mode is an interesting source of reusable infrastructure, but using its current output directly does not satisfy the desired experience. Compiling one independent object per managed assembly changes the compilation model and can forfeit whole-program optimizations or alter generic, reflection, metadata, and layout decisions. Such output is not necessarily representative of the final standard single-file NativeAOT build.
However, multifile compilation may still provide useful building blocks for incremental single-file compilation. One possible direction is:
1. Preserve the standard whole-program analysis and final output contract.
2. Partition generated code, metadata, dependency facts, or object fragments into cacheable intermediate files.
3. Rebuild only invalidated intermediate partitions.
4. Deterministically combine the intermediate files into the same single-file object or final executable shape that a clean compilation would produce.
In this design, multiple files are an internal incremental-build representation, not the user-visible compilation mode or final output contract.
A hybrid design could use assembly boundaries where they are safe and finer-grained partitions for very large assemblies. A retained compiler process or content-addressed cache might also reuse whole-program dependency facts that cannot be represented safely by independent assembly compilation.
This issue should evaluate these options rather than commit to the current multifile implementation or a retained-process prototype as the final architecture.
### Correctness and invalidation requirements
Incremental state must be keyed or invalidated by every semantic input, including:
- Primary assemblies and references
- Resources, substitutions, feature switches, reflection configuration, and trimming inputs
- Compiler, JIT, SDK, runtime, target, ABI, and instruction-set versions
- Optimization, inlining, scanning, preinitialization, devirtualization, and method-folding policy
- Generic dictionaries, virtual method facts, reflection metadata, interop stubs, and global tables
- Native debug information, exports, maps, source-link data, and other link-affecting outputs
- Relevant build properties and environment inputs
A changed partition must not reuse stale callers, metadata, relocation addends, unwind information, GC information, or global analysis results. Failed, interrupted, or corrupted incremental builds must never publish stale or partial output.
### Suggested success criteria
- Incremental compilation is the default for normal local NativeAOT builds after the first successful clean build.
- A standard source edit can use the incremental path without preserving an MVID, patching IL, or invoking ILC directly.
- Editing one project or method avoids recomputing unaffected whole-program work.
- The produced application remains representative of a clean production NativeAOT build for size, performance, and behavior analysis.
- Incremental and clean outputs are continuously compared in CI.
- Configuration, reference, resource, toolchain, and global-analysis changes invalidate the correct partitions.
- Clean fallback is automatic, deterministic, and clearly reported.
- Incremental state is bounded, versioned, evictable, isolated between requests, and recoverable after crashes.
- Windows, Linux, and macOS are covered on x64 and ARM64, followed by the remaining NativeAOT-supported targets.
- Performance is validated using at least one public large application in addition to private production workloads.
### Regression?
No. This is a long-standing scalability and developer-experience limitation of whole-program NativeAOT compilation.
### Related work
- Feasibility implementation and measurements: https://github.com/dotnet/runtime/pull/132962
- Existing unsupported assembly-granular mode: `IlcMultiModule=true`
> [!NOTE]
> This issue was prepared with GitHub Copilot assistance and reviewed by the submitting developer.
Contributor guide
Research direction
Start by reviewing the feasibility implementation and measurements in PR #132962, then inspect the existing unsupported IlcMultiModule=true mode as possible reusable infrastructure. Define an architecture that preserves whole-program NativeAOT output while caching valid intermediate work, with automatic invalidation and clean fallback. Done means representative incremental output, clear diagnostics, and coverage across the listed platforms and architectures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- build-system, developer-experience, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100