Client-IP rate-limit scope and burst allowance
@moonming is already working on this.
Since Jul 15, 2026.
- Dominant language
- Rust
- Stars
- 157
- Forks
- 32
- Avg merge
- 1h 25m
- Merged PRs (30d)
- 145
Description
Description
Rate limits key only on authenticated identities: api_key, model, team, member, team_member. There is no client-IP scope, no arbitrary header/XFF key, and no trusted-proxy / XFF-depth configuration for rate limiting. Counters are fixed-window with no burst allowance on top of a sustained rate.
Requested: an IP scope with trusted-proxy configuration, and an optional burst allowance layered over the sustained rate.
Why
Rate limiting is an abuse and denial-of-service control as much as a quota control, and identity-only keying cannot throttle an unauthenticated or pre-auth flood at the gateway.
Bursts matter because real client traffic is spiky: with a hard fixed window, operators must over-provision the limit to avoid false rejects, which defeats the point of setting one.
An edge/LB tier in front covers IP throttling in many deployments, which is why this is scored Low.
Priority
Low.
Prior art
| Product | Has it | Reference |
|---|---|---|
| LiteLLM | No | no IP scope |
| Kong AI Gateway | Yes | config.identifier: ip + trusted_ips at gateway level |
| Cloudflare AI Gateway | No | uniform, not IP-keyed |
| Portkey | No | no IP scope |
| Envoy AI Gateway | Partial | IP-keyed at the Envoy layer, not the AI layer |
| Bifrost | No | no IP scope |
| Helicone / TrueFoundry / Databricks AI Gateway / OpenRouter | Unknown | not researched |
Kong is the clearest prior art — note it keeps XFF trust at gateway level rather than per-limiter.
Surveyed 2026-07-15; every claim rests on a fetched docs/source page.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.