lablup / lablup/backend.ai

Make the rate limit window configurable at runtime

Open
#14,213 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Objective

Let operators change the rate limit window length without a code change, so the fixed window (BA-7648) can be shortened per deployment to reduce boundary bursts and the wait after a 429.

## Background

After BA-7648 the window is a hard-coded 15 minutes: _RATELIMIT_WINDOW in manager/api/rest/ratelimit/handler.py and web/ratelimit.py. The Valkey client already takes the window as an argument on every consume() call and RateLimitQuota takes it explicitly, so only the callers read a constant. Fixed-window limiters usually run on 1-second to 1-minute windows; a shorter window halves neither the limit nor the burst by itself, so changing it also changes what a keypair's rate_limit means (requests per window).

## Scope

- Add a manager configuration value for the window length (seconds), default 900, readable at runtime through the existing config provider.
- The manager middleware passes the configured value to consume() and to the X-RateLimit-Window header.
- The web server (BA-7603) must not need its own copy of the value: store the window in the per-user hash when the window opens, or derive it from the TTL, so both components report the same X-RateLimit-Window.
- A changed window applies to windows opened after the change; open windows keep their TTL.
- Update docs/manager/common-api/ratelimit.rst (window is configurable, default 15 minutes) and the keypair DTO descriptions that still say "requests per minute".

## Out of scope

- Converting stored keypair rate_limit values when the window changes.
- Per-keypair or per-user window lengths.

## Acceptance Criteria

- Setting the window to 60 s makes a new window expire after 60 s and X-RateLimit-Window report 60 from both the manager and the web server.
- Changing the value at runtime does not alter the TTL of an already open window.
- No _RATELIMIT_WINDOW constant remains in the manager or web server middleware.
- Unit tests cover the manager middleware with two different window values and the web server reporting the stored window.

## Related

- BA-7648 (fixed window counter, lablup/backend.ai#14211), BA-7603 (web server middleware, lablup/backend.ai#13771)

JIRA Issue: BA-7650

Contributor guide

Open the contributing guide

Research direction

Start with manager/api/rest/ratelimit/handler.py and web/ratelimit.py, then trace the existing config provider, consume() calls, RateLimitQuota, and per-user hash or TTL handling. Add the runtime window configuration and update docs/manager/common-api/ratelimit.rst plus the keypair DTO descriptions. Finish by covering two manager window values and web-server reporting, including unchanged TTLs for open windows.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.