Caching in Global Rate Limiting + more algorithms
- Dominant language
- C++
- Stars
- 28.9k
- Forks
- 5.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 437
Description
*Global Rate Limiting enhancements*: *Introduce local caching to reduce load on global rate limiting server, and reduce additional latency cost*
*Description*:
>Describe the desired behavior, what scenario it enables and how it
would be used.
When global rate limiting is enabled, each live request is sent to the global rate limiting service for evaluation, on whether to accept the request or not. This adds additional latency, and also requires the Global Rate Limiting Service to scale based on the RPS coming into the proxy.
Requirement 1: Can we add caching to Envoy main/worker thread itself, so that it doesn't impact a live request when making a rate limiting decision?
I have created a WASM filter written in RUST, which uses the Root context to sync with the Global Rate Limiting Service, and cache the results with a TTL. And the Root context shares cache with the Http Context, thereby preventing the worker thread from making a live request to the global rate limiting service. Maintaining the WASM filter will be an overhead, and it would be awesome if this could come as an option in the Global Rate Limiting Service in envoy.
Requirement 2: Domain teams might be interested in using different algorithms based on their needs, can we also support FIXED WINDOW and SLIDING WINDOW algorithms ?
[optional *Relevant Links*:]
>Any extra documentation required to understand the issue.
Contributor guide
Assessment
This issue has not been assessed yet.