dotnet / dotnet/runtime

[API Proposal]: RateLimiterLease Metadata - ChainedRateLimiter InnerLeases

Open
#130,114 1 comment 0 reactions 0 assignees View on GitHub
api-suggestion area-System.Threading
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Background and motivation

When using a `ChainedRateLimiter` it's not possible to reliably determine which inner `RateLimiter` has blocked acquisition.

My specific use case is a "changed notification" event processing scenario, where the notification only contains a key and it is very common for several notifications to arrive roughly simultaneously.

```
static readonly ConcurrencyLimiterOptions _concurrencyLimiterOptions = new()
{
QueueProcessingOrder = QueueProcessingOrder.NewestFirst,
PermitLimit = 1,
QueueLimit = 1,
};

static readonly TokenBucketRateLimiterOptions _tokenBucketRateLimiterOptions = new()
{
// 1 per minute, bursting to 10 per minute
AutoReplenishment = true,
QueueLimit = 1,
QueueProcessingOrder = QueueProcessingOrder.NewestFirst,
ReplenishmentPeriod = TimeSpan.FromMinutes(1),
TokenLimit = 10,
TokensPerPeriod = 1,
};

readonly PartitionedRateLimiter _itemLimiter = PartitionedRateLimiter.Create(
static key => new(key, static key => RateLimiter.CreateChained([
new ConcurrencyLimiter(_concurrencyLimiterOptions),
new TokenBucketRateLimiter(_tokenBucketRateLimiterOptions),
])
));
```

Events for each key are processed one at a time, and if an event arrives before the previous notification has been processed, queue only a single notifications. Any rejections here is non-exceptional and expected.

Additionally, a `TokenBuckedRateLimiter` is chained as a circuit breaker to detect/limit runaways. Any rejections indicate a real issue and should be surfaced separately.

Other more typical scenarios include having a tiered rate limits, such a instance/tenant/user.

### API Proposal

```csharp
namespace System.Threading.RateLimiting;

public static partial class MetadataName
{
public static readonly MetadataName> InnerLeases { get; }
}
```

### API Usage

```csharp
var lease = await _itemLimiter.AcquireAsync(key);
if (!lease.IsAcquired)
{
if (lease.TryGetMetadata(MetadataName.InnerLeases, out var leases)
&& leases.Count < 2))
{
return Ok("newer notification queued");
}
return Error("Rate limit reached, check for runaway!");
}
```

### Alternative Designs

1. Manually chain `RateLimiter` instances. This is what the [ASP.NET Core middleware does](https://github.com/dotnet/aspnetcore/blob/b12b77b241f0a093d53508c3cb2084860bd5339d/src/Middleware/RateLimiting/src/RateLimitingMiddleware.cs#L19-L20) does for global vs endpoint rate limiting.
2. Rely on undocumented/coincidental behavior - eg. currently it seems like only `ConcurrencyLimiter` sets `ReasonCode` metadata and only ever to `"Queue limit reached"`, so if that is present and depending on the configuration of the other limiters, it can be inferred that it was the concurrency limit was tripped in certain circumstances. Fragile.
3. Just have an index instead of IReadOnlyList - `RejectingLimiterIndex`.
4. Add a "name" string to each entry in the chain and plumb that through - eg. `RejectingLimiterName`.
```
RateLimitPartition.CreateChain([
("global", globalLimiter),
("tenant", tenantLimiter),
("user", userLimiter)
]);

lease.TryGetMetadata(RateLimitMetadataName.RejectingLimiter, out var limiterName);

```

### Risks

Potentially limiting future optimizations around combining the implementations of various combinations of rate limiters.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ChainedRateLimiter, RateLimiterLease, and MetadataName entry points, then compare the ASP.NET Core RateLimitingMiddleware example linked in the issue. Assess how InnerLeases would be exposed without limiting future optimizations; done means the proposal has a decided API shape and usage can distinguish the rejecting limiter.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.