dotnet / dotnet/yarp

Pending request queue size limit

Open
#394 4 comments 0 reactions 0 assignees View on GitHub
Type: Idea
Dominant language
C#
Stars
9.6k
Forks
933
Avg merge
12d 18h
Merged PRs (30d)
2

Description

When the maximum number of HTTP 1.1 connections or the maximum of HTTP/2 active streams is reached on a destination, all subsequent requests are put on a queue inside `SocketsHttpMessageHandler` where they will wait until either they get processed by the destination or timeout happens.

There are 2 disadvantages in this approach:
1. Allowing an indefinite number of request gets queued can eventually overload destination if no other throttling mechanism is in place
2. If destination became unhealthy, all pending request will eventually fail due to a timeout, but it will happen only after some time delaying the client reaction to these failures.

It's proposed to add an optional **pending request queue size** parameter to the proxy HTTP client configuration limiting the number of queued requests. Once the queue is full, all subsequent requests to the same destination will be rejected with an error.

**NOTE:** It probably requires adding a new feature to `SocketsHttpMessageHandler`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.