envoyproxy / envoyproxy/envoy

Caching in Global Rate Limiting + more algorithms

Open
#39,411 8 comments 0 reactions 0 assignees View on GitHub
area/ratelimit enhancement no stalebot
Dominant language
C++
Stars
28.9k
Forks
5.6k
Avg merge
1d 20h
Merged PRs (30d)
437

Description

*Global Rate Limiting enhancements*: *Introduce local caching to reduce load on global rate limiting server, and reduce additional latency cost*

*Description*:
>Describe the desired behavior, what scenario it enables and how it
would be used.
When global rate limiting is enabled, each live request is sent to the global rate limiting service for evaluation, on whether to accept the request or not. This adds additional latency, and also requires the Global Rate Limiting Service to scale based on the RPS coming into the proxy.

Requirement 1: Can we add caching to Envoy main/worker thread itself, so that it doesn't impact a live request when making a rate limiting decision?

I have created a WASM filter written in RUST, which uses the Root context to sync with the Global Rate Limiting Service, and cache the results with a TTL. And the Root context shares cache with the Http Context, thereby preventing the worker thread from making a live request to the global rate limiting service. Maintaining the WASM filter will be an overhead, and it would be awesome if this could come as an option in the Global Rate Limiting Service in envoy.

Requirement 2: Domain teams might be interested in using different algorithms based on their needs, can we also support FIXED WINDOW and SLIDING WINDOW algorithms ?

[optional *Relevant Links*:]
>Any extra documentation required to understand the issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.