[API Proposal]: RateLimiterLease Metadata - ChainedRateLimiter InnerLeases
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Background and motivation
When using a `ChainedRateLimiter` it's not possible to reliably determine which inner `RateLimiter` has blocked acquisition.
My specific use case is a "changed notification" event processing scenario, where the notification only contains a key and it is very common for several notifications to arrive roughly simultaneously.
```
static readonly ConcurrencyLimiterOptions _concurrencyLimiterOptions = new()
{
QueueProcessingOrder = QueueProcessingOrder.NewestFirst,
PermitLimit = 1,
QueueLimit = 1,
};
static readonly TokenBucketRateLimiterOptions _tokenBucketRateLimiterOptions = new()
{
// 1 per minute, bursting to 10 per minute
AutoReplenishment = true,
QueueLimit = 1,
QueueProcessingOrder = QueueProcessingOrder.NewestFirst,
ReplenishmentPeriod = TimeSpan.FromMinutes(1),
TokenLimit = 10,
TokensPerPeriod = 1,
};
readonly PartitionedRateLimiter _itemLimiter = PartitionedRateLimiter.Create(
static key => new(key, static key => RateLimiter.CreateChained([
new ConcurrencyLimiter(_concurrencyLimiterOptions),
new TokenBucketRateLimiter(_tokenBucketRateLimiterOptions),
])
));
```
Events for each key are processed one at a time, and if an event arrives before the previous notification has been processed, queue only a single notifications. Any rejections here is non-exceptional and expected.
Additionally, a `TokenBuckedRateLimiter` is chained as a circuit breaker to detect/limit runaways. Any rejections indicate a real issue and should be surfaced separately.
Other more typical scenarios include having a tiered rate limits, such a instance/tenant/user.
### API Proposal
```csharp
namespace System.Threading.RateLimiting;
public static partial class MetadataName
{
public static readonly MetadataName> InnerLeases { get; }
}
```
### API Usage
```csharp
var lease = await _itemLimiter.AcquireAsync(key);
if (!lease.IsAcquired)
{
if (lease.TryGetMetadata(MetadataName.InnerLeases, out var leases)
&& leases.Count < 2))
{
return Ok("newer notification queued");
}
return Error("Rate limit reached, check for runaway!");
}
```
### Alternative Designs
1. Manually chain `RateLimiter` instances. This is what the [ASP.NET Core middleware does](https://github.com/dotnet/aspnetcore/blob/b12b77b241f0a093d53508c3cb2084860bd5339d/src/Middleware/RateLimiting/src/RateLimitingMiddleware.cs#L19-L20) does for global vs endpoint rate limiting.
2. Rely on undocumented/coincidental behavior - eg. currently it seems like only `ConcurrencyLimiter` sets `ReasonCode` metadata and only ever to `"Queue limit reached"`, so if that is present and depending on the configuration of the other limiters, it can be inferred that it was the concurrency limit was tripped in certain circumstances. Fragile.
3. Just have an index instead of IReadOnlyList - `RejectingLimiterIndex`.
4. Add a "name" string to each entry in the chain and plumb that through - eg. `RejectingLimiterName`.
```
RateLimitPartition.CreateChain([
("global", globalLimiter),
("tenant", tenantLimiter),
("user", userLimiter)
]);
lease.TryGetMetadata(RateLimitMetadataName.RejectingLimiter, out var limiterName);
```
### Risks
Potentially limiting future optimizations around combining the implementations of various combinations of rate limiters.
Contributor guide
Research direction
Start by reviewing the ChainedRateLimiter, RateLimiterLease, and MetadataName entry points, then compare the ASP.NET Core RateLimitingMiddleware example linked in the issue. Assess how InnerLeases would be exposed without limiting future optimizations; done means the proposal has a decided API shape and usage can distinguish the rejecting limiter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100