dotnet / dotnet/runtime

IIS in-process hosting returns HTTP 500 after runtime-async enablement

Open
#126,925 7 comments 0 reactions 1 assignee Claimed by @ericstj View on GitHub
area-Extensions-Logging
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

## Description

After enabling runtime-async unconditionally in System.Private.CoreLib (#126680, #126594), IIS in-process hosted ASP.NET Core applications return HTTP 500 for all requests. The application starts successfully (ANCM logs confirm "started the coreclr in-process successfully") but every subsequent HTTP request fails with 500 Internal Server Error — the managed request pipeline never produces a successful response.

This was discovered via a codeflow PR in dotnet/aspnetcore ([#66239](https://github.com/dotnet/aspnetcore/pull/66239)) that brings in runtime changes from commit range [`655c517c...ce3e7165`](https://github.com/dotnet/runtime/compare/655c517ccccad501dddac7c296815fabb2c8f1ad...ce3e7165836e64efbfe6f7a874991715b40e28bf).

## Reproduction

**Failing test:** `IIS.NewShim.FunctionalTests.StartupTests.StartsWithPortableAndBootstraperExe`

**Build:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1378813&view=results

**Helix job:** `db741402-6d7f-4dfc-bea7-30db6cf73900`, work item `IIS.NewShim.FunctionalTests--net11.0`

The test deploys an ASP.NET Core app (`InProcessWebSite`) to IIS using the in-process hosting model, then sends GET requests to `/HelloWorld` expecting a 200 response with body "Hello World". Instead, every request returns 500 Internal Server Error.

## Key observations from the Helix logs

The ANCM (ASP.NET Core Module) logs show the app starts without errors:

```
[aspnetcorev2_inprocess.dll] In-process callbacks set
[aspnetcorev2_inprocess.dll] Event Log: 'Application '...' started the coreclr in-process successfully.'
```

But all HTTP requests return 500:

```
StatusCode: 500, ReasonPhrase: 'Internal Server Error', Version: 1.1
Server: Microsoft-IIS/10.0
```

The test retries 10 times over ~11 seconds, receiving 500 each time, then fails:

```
System.InvalidOperationException : Didn't get response that satisfies predicate after 10 retries
```

No exception or error message is logged by the application itself — the 500 appears to come from the request pipeline silently failing to complete.

## Suspected cause

The runtime diff between the working and broken builds includes two commits that enable runtime-async unconditionally:

- **714d206ca6** — [Enable runtime-async in CoreCLR System.Private.CoreLib (#126594)](https://github.com/dotnet/runtime/commit/714d206ca6)
- **ddba7854ea** — [Enable runtime-async unconditionally in System.Private.CoreLib (CoreCLR and NativeAOT) (#126680)](https://github.com/dotnet/runtime/commit/ddba7854ea)

IIS in-process hosting has a complex interaction between the native ANCM module and managed async code — the ASP.NET Core request pipeline runs async methods within the IIS worker process, with continuations that cross native/managed boundaries. A change in how async continuations are scheduled or how tasks complete at the JIT level could cause the request handler to fail without completing the response.

## Suggested investigation

1. Confirm by running the failing test with `DOTNET_JitEnableRuntimeAsync=0` — if the test passes with runtime-async disabled, this confirms the root cause.
2. Investigate the interaction between runtime-async and the IIS in-process hosting model's `IHttpApplication.ProcessRequestAsync` / `IISHttpContext` pipeline.

## Environment

- **OS:** Windows Server (Helix agent: `Microsoft Windows NT 10.0.26100.0`)
- **Runtime:** .NET 11.0 Preview 4 (`11.0.0-preview.4.26210.110`)
- **aspnetcore PR:** https://github.com/dotnet/aspnetcore/pull/66239
- **Build:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1378813

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.