Make the rate limit window configurable at runtime
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Objective
Let operators change the rate limit window length without a code change, so the fixed window (BA-7648) can be shortened per deployment to reduce boundary bursts and the wait after a 429.
## Background
After BA-7648 the window is a hard-coded 15 minutes: _RATELIMIT_WINDOW in manager/api/rest/ratelimit/handler.py and web/ratelimit.py. The Valkey client already takes the window as an argument on every consume() call and RateLimitQuota takes it explicitly, so only the callers read a constant. Fixed-window limiters usually run on 1-second to 1-minute windows; a shorter window halves neither the limit nor the burst by itself, so changing it also changes what a keypair's rate_limit means (requests per window).
## Scope
- Add a manager configuration value for the window length (seconds), default 900, readable at runtime through the existing config provider.
- The manager middleware passes the configured value to consume() and to the X-RateLimit-Window header.
- The web server (BA-7603) must not need its own copy of the value: store the window in the per-user hash when the window opens, or derive it from the TTL, so both components report the same X-RateLimit-Window.
- A changed window applies to windows opened after the change; open windows keep their TTL.
- Update docs/manager/common-api/ratelimit.rst (window is configurable, default 15 minutes) and the keypair DTO descriptions that still say "requests per minute".
## Out of scope
- Converting stored keypair rate_limit values when the window changes.
- Per-keypair or per-user window lengths.
## Acceptance Criteria
- Setting the window to 60 s makes a new window expire after 60 s and X-RateLimit-Window report 60 from both the manager and the web server.
- Changing the value at runtime does not alter the TTL of an already open window.
- No _RATELIMIT_WINDOW constant remains in the manager or web server middleware.
- Unit tests cover the manager middleware with two different window values and the web server reporting the stored window.
## Related
- BA-7648 (fixed window counter, lablup/backend.ai#14211), BA-7603 (web server middleware, lablup/backend.ai#13771)
JIRA Issue: BA-7650
Contributor guide
Research direction
Start with manager/api/rest/ratelimit/handler.py and web/ratelimit.py, then trace the existing config provider, consume() calls, RateLimitQuota, and per-user hash or TTL handling. Add the runtime window configuration and update docs/manager/common-api/ratelimit.rst plus the keypair DTO descriptions. Finish by covering two manager window values and web-server reporting, including unchanged TTLs for open windows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100