dotnet / dotnet/aspnetcore

Reduce ArrayPool.Shared locking in Kestrel's HTTP/3 path

Open
#66,102 0 comments 0 reactions 0 assignees View on GitHub
area-networking HTTP3
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

I have noticed a different behaviour of GetSpan() between HTTP/2 vs HTTP/3 in Kestrel when it comes to renting memory:
[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http2.Http2OutputProducer.GetSpan()](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Servers/Kestrel/Core/src/Internal/Http2/Http2OutputProducer.cs#L481) vs
[Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http3.Http3OutputProducer.GetSpan()](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Servers/Kestrel/Core/src/Internal/Http3/Http3OutputProducer.cs#L213)

HTTP/2 uses MemoryPool (PinnedBlockMemoryPool by default): [SocketTransportOptions.cs](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Servers/Kestrel/Transport.Sockets/src/SocketTransportOptions.cs#L169).
PinnedBlockMemoryPool has its own pool of cached memory blocks that it re-uses when Rent() is called, provided that the requested size does not exceed its MaxBufferSize which is 4 KB [PinnedBlockMemoryPool.cs](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Shared/Buffers.MemoryPool/PinnedBlockMemoryPool.cs#L18). PinnedBlockMemoryPool keeps references to its memory blocks in System.Collections.Concurrent.ConcurrentQueue, which is lock-free on its fast path (uses atomic operations via Interlocked, volatile reads/writes with memory fences/barriers to guarantee visibility and ordering across CPUs).
Also, the MemoryPool instance is local to Kestrel ( [here](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Servers/Kestrel/Transport.Sockets/src/SocketConnectionContextFactory.cs#L50) and [here](https://github.com/dotnet/aspnetcore/blob/baa6b294e728e6171378b4e8c52e42e7c4d4ed63/src/Servers/Kestrel/Transport.Sockets/src/SocketConnectionContextFactory.cs#L65) ) and not shared across entire .NET process.

HTTP/3 uses ArrayPool.Shared, which is a [singleton](https://github.com/dotnet/runtime/blob/c57884892e0b1e3b63c75ffdc0f865a0db313e61/src/libraries/System.Private.CoreLib/src/System/Buffers/ArrayPool.cs#L23) instance of [SharedArrayPool](https://github.com/dotnet/runtime/blob/c57884892e0b1e3b63c75ffdc0f865a0db313e61/src/libraries/System.Private.CoreLib/src/System/Buffers/SharedArrayPool.cs#L22). SharedArrayPool has a more complex implementation with great heuristics. For example, it uses separate buckets for different memory block sizes. It can use thread-local storage. It prefers renting out recently returned memory blocks first which might still be on CPU cache lines, thus helping cache locality etc.
At the same time, at least in my workload, I can see that ArrayPool.Shared (System.Buffers.SharedArrayPool.Rent() method) spends most of its time (i.e. 82%) in locking, i.e. libcoreclr.so!JIT_MonEnter_Portable and libcoreclr.so!JIT_MonExit_Portable. I suspect that it might be happening because ArrayPool.Shared is shared across entire .NET process.

Question: would it improve Http3OutputProducer.GetSpan() performance if Kestrel's HTTP/3 path was also using its own memory pool (either PinnedBlockMemoryPool or maybe a dedicated instance of ArrayPool)?

A potential code change needed for allowing HTTP/3 to use its own MemoryPool:
[http3_memorypool.patch](https://github.com/user-attachments/files/26428871/http3_memorypool.patch)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.