JIT: Improve tier1 OSR code quality for long-running runtime-async methods
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Motivation
Long-running async methods frequently contain request-processing or I/O loops. With runtime async, these methods can remain logically active for the lifetime of a connection, suspending and resuming many times without completing the invocation.
This makes Tier1 OSR code quality particularly important. Ordinary tier promotion benefits subsequent invocations, but a long-running invocation needs OSR to transition away from Tier0. In the ASP.NET benchmarks investigated here, the hot versions of these loops were Tier1 OSR bodies, not ordinary Tier1 bodies.
We should consider making this a focus area for .NET 12: OSR is not just a startup transition for these methods. Its code can become the steady-state implementation for a substantial part of the workload.
### Examples in ASP.NET Core
Three Kestrel methods illustrate this pattern:
- [`SocketConnection.DoSend`](https://github.com/dotnet/aspnetcore/blob/6c010a60b7aaa8297b2807681faeaa711f79c34d/src/Servers/Kestrel/Transport.Sockets/src/Internal/SocketConnection.cs#L266-L321): a `while (true)` loop that awaits pipe reads and socket sends.
- [`SocketConnection.DoReceive`](https://github.com/dotnet/aspnetcore/blob/6c010a60b7aaa8297b2807681faeaa711f79c34d/src/Servers/Kestrel/Transport.Sockets/src/Internal/SocketConnection.cs#L133-L263): a connection-lifetime loop that awaits socket readiness, receives data, and flushes the input pipe.
- [`HttpProtocol.ProcessRequests`](https://github.com/dotnet/aspnetcore/blob/6c010a60b7aaa8297b2807681faeaa711f79c34d/src/Servers/Kestrel/Core/src/Internal/Http/HttpProtocol.cs#L646-L673): a keep-alive request loop containing a nested read/parse loop, followed by application processing and response completion.
These are not necessarily expensive individual invocations by call count. Their importance comes from the amount of work done inside one invocation and across its async resumptions.
### Observed problem
In the Linux Arm64 JSON benchmark, the normal `DoSend` compilation produced Instrumented Tier0 followed by Tier1 OSR. The OSR compilation received Dynamic PGO data.
One dump had an entry count of approximately 1 and approximately 245,000 loop backedges. That entry count is consistent with a single transfer into the long-running OSR invocation; it does not mean the loop executes only once.
The resulting high loop frequencies relative to entry make call sites inside the loop appear extremely hot to the inliner. In this workload, the Tier1 OSR compilation performed extensive inlining. This increased code size, frame size, and the number of suspension paths.
For one matched `DoSend` comparison, with the same experimental `Pipe.ReadAsync` source restructuring in both configurations:
| Metric | Normal Tier1 OSR | `AggressiveOptimization` FullOpts |
|---|---:|---:|
| Native code size | 6,636 bytes | 2,104 bytes |
| Instruction count | 1,663 | 526 |
| Stack frame | 544 bytes | 152 bytes |
| Successful inlines | 190 | 43 |
| Suspension states | 4 | 2 |
| Continuation data size | 232 bytes | 168 bytes |
Exact numbers vary between Dynamic PGO captures, but the large difference in compilation shape was clear.
`AggressiveOptimization` is not an isolated test of OSR machinery: it bypasses tiering and Dynamic PGO as well. The comparison therefore implicates the overall OSR/PGO/inlining strategy, rather than proving that OSR entry or continuation resumption itself accounts for the cost.
### Performance evidence
In earlier adjacent Arm64 JSON comparisons, applying `AggressiveOptimization` to these three methods improved throughput (measured by RPS) by approximately 4%. Isolated experiments showed improvements for `ProcessRequests` and `DoSend`.
Continuation capture was also contributing unnecessary work. Separate JIT experiments removed falsely live struct temporaries and reduced continuation copying. However, reducing capture alone did not consistently reproduce the gains from `AggressiveOptimization`.
This suggests that continuation size is only part of the problem. The larger inline graph, frame size, spills, and duplicated suspension/control-flow paths also warrant investigation. We have not isolated their individual contributions.
### Suggested .NET 12 investigation
- Add representative long-lived runtime-async loops to steady-state performance and code-quality coverage, including execution across repeated suspension and resumption.
- Examine how OSR entry counts and loop frequencies influence Dynamic PGO inlining profitability.
- Account for the code-size and suspension-state costs of async inlining, not just the apparent hotness of the call site.
- Compare Tier1 OSR, ordinary Tier1 where reachable, and non-tiered FullOpts code using equivalent workloads and carefully controlled profiling inputs.
- Track frame size, spills, continuation capture, and suspension-path duplication alongside throughput.
The goal should be good steady-state code for these common async patterns without requiring applications or frameworks to add `AggressiveOptimization` annotations that give up Dynamic PGO.
> [!NOTE]
> Drafted with GitHub Copilot from the benchmark and JIT-codegen investigation.
Contributor guide
Research direction
Start with the referenced Kestrel methods in SocketConnection.cs and HttpProtocol.cs, then reproduce the Linux Arm64 JSON benchmark comparisons for Tier1 OSR, ordinary Tier1, and FullOpts. Done means identifying how OSR/PGO inlining affects code size, frames, spills, continuation capture, and suspension paths, with representative steady-state coverage and measured results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- compilers, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100