dotnet / dotnet/aspnetcore

Additional Events for RateLimitOptions

Open
#45,705 1 comment 0 reactions 1 assignee Claimed by @BrennanConroy View on GitHub
api-suggestion area-middleware feature-rate-limit
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 5h
Merged PRs (30d)
276

Description

## Background and Motivation

We have memory intensive application hosted in Kubernetes, this application's memory is directly proportional to the number of requests it can handle. We were using `Semaphore` to limit the concurrency and create internal queues for processing.

Now that .Net7 has a `ConcurrencyLimiter` we started investigating on the same, the idea (although non-proven) was to turn off the `Readiness` probe once the concurrency limit had reached so that the traffic wouldn't flow to that instance, once the limit is replenished we could turn on the `Readiness` probe to start allowing the traffic back to the pod.

Unfortunately, the RateLimiter option has only one event, and that occurs on the rejection of the request since the limit was breached.

```csharp
public Func? OnRejected { get; set; }
```
Even if we consume this event to turn off the probe, there is no way to bring the readiness probe back as there is no apparent way to check if the limits are replenished.

```csharp
builder.Services.AddRateLimiter(_ =>
{
_.AddConcurrencyLimiter(policyName: "LimiterPolicy", options =>
{
options.PermitLimit = 10;
// Only allow 10 request every 10 seconds
options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
// Only queue 3 requests when we go over that limit
options.QueueLimit = 3;
});
_.OnRejected = async (context, token) =>
{
context.HttpContext.Response.StatusCode = 429;
ConcurrencyLimitHolder.HasLimitReached = true;
};
});

///in the probe controller
[HttpGet("ready")]
public IActionResult GetReadiness()
{
if (ConcurrencyLimitHolder.HasLimitReached)
return BadRequest();

return Ok();
}
```

This isn't very useful, because the instance has already started rejecting the requests and the clients would have to retry, but the chances are that it would end up in the same instance.

## Proposed API
An event triggered as soon as the rate limit is replenished/reset
```csharp
public sealed class RateLimiterOptions
{
public Func? OnReplinished { get; set; }
}
```
## Usage Examples
using the same example as above

```csharp
builder.Services.AddRateLimiter(_ =>
{
_.AddConcurrencyLimiter(policyName: "LimiterPolicy", options =>
{
options.PermitLimit = 10;
// Only allow 10 request every 10 seconds
options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
// Only queue 3 requests when we go over that limit
options.QueueLimit = 3;
});
_.OnRejected = async (context, token) =>
{
context.HttpContext.Response.StatusCode = 429;
ConcurrencyLimitHolder.HasLimitReached = true;
};
_.OnReplenished = async(context,token) =>
{
ConcurrencyLimitHolder.HasLimitReached = false;
}
});

///in the probe controller
[HttpGet("ready")]
public IActionResult GetReadiness()
{
if (ConcurrencyLimitHolder.HasLimitReached)
return BadRequest();

return Ok();
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.