envoyproxy / envoyproxy/gateway

local ratelimit: max_dynamic_descriptors only on the HCM filter, route-level limiters track 20 distinct values

Open
#9,973 0 comments 0 reactions 1 assignee Claimed by @andreeapid View on GitHub
kind/bug
Dominant language
Go
Stars
3k
Forks
864
Avg merge
2d 2h
Merged PRs (30d)
140

Description

### Description

A `rateLimit.local` rule with a `Distinct` selector (`sourceCIDR`, header or query parameter) tracks only the 20 most recently seen values per rule. The 21st value evicts the oldest bucket, and an evicted client comes back with a full bucket, so a per-client limit on a public listener is not a bound.

### Root cause

`internal/xds/translator/local_ratelimit.go` sets `MaxDynamicDescriptors: 10000` in `patchHCM`, on the HCM-level filter that its own comment describes as empty ("The real configuration is done at the route level"). The route-level `LocalRateLimit` built in `buildRouteLocalRateLimits` carries the descriptors but not the field.

Envoy resolves the per-route config as a whole (`resolveMostSpecificPerFilterConfig` in `source/extensions/filters/http/local_ratelimit/local_ratelimit.cc`), and each `FilterConfig` reads `max_dynamic_descriptors` from its own proto with a default of 20. Nothing is inherited from the HCM filter, so the 10000 never applies to any route. Eviction happens in `DynamicDescriptor::addorGetDescriptor` (`local_ratelimit_impl.cc`, logged at trace level as `lru_size(20) overflow`).

### Repro

```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: per-ip
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: backend
rateLimit:
local:
rules:
- clientSelectors:
- sourceCIDR:
type: Distinct
value: 0.0.0.0/0
limit:
requests: 1
unit: Minute
```

With a `ClientTrafficPolicy` trusting one `X-Forwarded-For` hop, so one host can present many client IPs:

1. IP1: 200, then 429 (bucket exhausted, as expected)
2. IP2 to IP21: 200 each
3. IP1 again: 200. Expected 429; its bucket was evicted and recreated full.

The `local-ratelimit-distinct` xds-ir fixture shows the same thing: the listener filter has `maxDynamicDescriptors: 10000`, the route-level `typed_per_filter_config` has no such field.

### Versions

v1.8.3, v1.9.1 and `main` (27cbfab84). Present since Distinct support for local rate limits (#5319, v1.4.0).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.