api7 / api7/aisix

P0-5: Distributed (Redis-backed) rate limiting

Open
#47 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

gap-with-litellm P0
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Problem

crates/aisix-ratelimit/src/limiter.rs keeps state in a DashMap — process-local. The moment a deployment runs more than one DP replica (which is the default for any non-trivial setup), each replica enforces its own counter. Effective rate limit becomes N × configured limit where N = replica count. Hard blocker for prod multi-replica deployment.

Scope

DP (moonming/ai-gateway)
  • New RedisLimiter impl behind the same Limiter trait (already abstracted at limiter.rs:1-2).
  • Use atomic Redis Lua scripts for fixed-window counter (RPM/RPD/TPM/TPD) — single round-trip per check + commit. Same SipHash-fingerprinted key shape used today.
  • Concurrency limiter: probably keep local (cluster-wide concurrency tracking is complex; per-replica concurrency cap is usually fine).
  • Add rate_limit: config: backend: memory | redis + redis URL/auth (reuse aisix-cache/src/redis.rs connection logic).
  • Algorithm: keep fixed-window for v1 to mirror existing semantics; sliding-window can be a follow-up.
CP (api7/api7ee-3-control-plane)
  • No CRUD changes (rate_limit is per-api_key field on ai_api_key, #2625 already lists it as env-scoped).
  • Optional: expose Redis backend choice in DP config push.
Dashboard UI (api7/AISIX-Cloud)
  • Environment Settings page: "Rate limit backend" toggle (memory / Redis) + Redis URL field.
e2e
  • Spin two DP replicas in compose, hit them round-robin, assert combined RPM cap is enforced (not 2×).

Estimate

DP 3d, CP 0d, UI 0.5d, e2e 1d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/aisix-ratelimit/src/limiter.rs and the Redis connection logic in aisix-cache/src/redis.rs, then review the DP configuration and existing rate-limit behavior. Check the Environment Settings page and the compose-based e2e setup for the UI and multi-replica requirements. Done means Redis-backed fixed-window limits enforce the combined cap across replicas, with configuration and coverage for the requested paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, rust
Domain
backend, databases, distributed-systems, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.