dotnet / dotnet/aspnetcore

Investigate performance of large writes on Kestrel

Open
#31,110 4 comments 3 reactions 1 assignee Assigned to @davidfowl View on GitHub
area-networking Perf
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

When testing writing a 1MB chunked response, we spend a considerable amount of time copying buffers (this profile is on windows):

![image](https://user-images.githubusercontent.com/95136/111912151-9bf6d500-8a25-11eb-87b6-31ea499fd026.png)

The above shows:
- 20% of the time copying into kernel buffers
- 10% of the time copying data from the user's buffer into Kestrel's buffer using 4K blocks
- 2% of the time in GetSpan

Here's what the flow looks like for HTTP/1.1 connections:

## HTTPS

- Socket.SendAsync(...) -> copies buffers into kernel
- PipeWriter.WriteAsync(...) -> copies buffer then flushes
- SslStream.WriteAsync(...) -> copies and encrypts individual buffers
- StreamPipeWriter.FlushAsync() ~copies buffer then flushes~ (copy removed in .NET 6)
- ConcurrentPipeWriter.WriteAsync() (usually passthrough)
- Http1OutputProducer.WriteAsync(buffer)
- HttpResponse.Body.WriteAsync(buffer)

## HTTP

- Socket.SendAsync(...) -> copies buffers into kernel buffer
- PipeWriter.WriteAsync(...) -> copies buffer then flushes
- ConcurrentPipeWriter.WriteAsync() (usually passthrough)
- Http1OutputProducer.WriteAsync(buffer)
- HttpResponse.Body.WriteAsync(buffer)

Packet sizes on the wire look good for both TLS and non-TLS connections:

Non-TLS

![image](https://user-images.githubusercontent.com/95136/111942329-50c5dc00-8a90-11eb-869d-8cb08ec66eea.png)

TLS

![image](https://user-images.githubusercontent.com/95136/111942358-5ae7da80-8a90-11eb-9107-cf1a22324cc9.png)

Code sample:

```C#
using System;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var host = new HostBuilder().ConfigureWebHostDefaults(host =>
{
host.Configure(app =>
{
var s = Encoding.UTF8.GetBytes(new string('A', 1024 * 512)).AsMemory();

async Task Hello(HttpContext context)
{
await context.Response.StartAsync();

var source = s;

var memory = context.Response.BodyWriter.GetMemory();

source[..memory.Length].CopyTo(memory);

context.Response.BodyWriter.Advance(memory.Length);

source = source[memory.Length..];

// context.Response.ContentLength = s.Length;
await context.Response.Body.WriteAsync(source);
}

app.UseRouting();

app.UseEndpoints(routes =>
{
routes.MapGet("/", Hello);
});
});
}).Build();

await host.RunAsync();
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.