dotnet / dotnet/aspnetcore

HTTP/2 multiplexing performance issues

Open
#32,903 4 comments 0 reactions 0 assignees View on GitHub
area-networking bug feature-iis Perf
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

_Update: I have only been able to reproduce the behavior with requests blocking for several seconds under certain network conditions. So this might only partially be a problem with ASP.NET Core._

Writing to the response body stream with ``HttpContext.Response.Body.WriteAsync`` blocks other concurrent requests.

When server code writes data to the response stream in multiple writes - e.g. a large download - and the client calls the same code for more data, these calls will take an unexpectedly long time to finish.

* This is mostly a problem with IIS hosting. Kestrel performs much better, although not as fast as with HTTP/1.1.
* This is not _only_ a problem with IIS. I cannot reproduce it with .NET Framework.
* This is only reproducible with HTTP/2.
* A workaround in IIS is to either use ``Response.BodyWriter.WriteAsync`` or to issue a ``Response.Body.FlushAsync`` after each write. This will result in response times comparable to those with Kestrel, although not as fast as with HTTP/1.1.
* I can reproduce this on .NET Core 3.1 and 5.0.
* I cannot reproduce the worst observed behavior under all network conditions.

Here is a small repro for testing. Consider the following server code, wich I _think_ is a supported way of producing response content.
```csharp
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapGet("/data", async context =>
{
int length = int.Parse(context.Request.Query["length"]);
int written = 0;
while (written < length)
{
var chunk = new byte[Math.Min(length - written, 4096)];
await context.Response.Body.WriteAsync(chunk.AsMemory());
written += chunk.Length;
}
});
});
```

The ``FileResultExecutorBase`` uses a similar approach for writing the response, so the behavior can be reproduced with MVC actions returning ``FileStreamResult`` or ``FileContentResult``.

Here is some client code for testing in a browser:
```html

function fetchData() {
for (let i = 0; i < 5; i++) {
fetch("data?length=102400")
.then(response => response.arrayBuffer()
.then(content => console.log(content.byteLength)));
}
}

Download 1GB
Fetch 100KB
```

**Steps to reproduce**

1. Click _Download 1GB_ to start downloading data.
2. Click _Fetch 100KB_ to start five concurrent requests for a smaller amount of data.

**Observed behavior**

The five requests for the 100KB take an unreasonably long time to finish, not simply explainable by the bandwidth usage of the large download. Over a 100Mbit network with a 17ms ping round trip time, the first one takes a couple of seconds, and the last around 7 seconds, in Edge. In FireFox, they don't seem to finish until I cancel the large download.

With HTTP/1.1, they finish within a few tens of ms. In Kestrel (HTTP/2), they finish within a few hundred milliseconds.

_Using a 80KB buffer in the repro has the same results._

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.