apache / apache/pinot

[Feature] Create a server-level consumer rate limit

Open
#8,637 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

follow up on #6291. we now have a way to throttle on a per-table basis. but we don't have a way to limit the global rate for the server.

I don't have a good idea on how to limit on a server level or how to distribute the rate limit fairly to each realtime table. but this would be a good feature to have to safeguard, otherwise ops will have to come up with clever configuration balance for each table.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing issue #6291 and the existing per-table throttling implementation referenced in this issue. Define how a server-wide consumer rate should be enforced and fairly distributed across realtime tables, then document measurable behavior for the global limit and its interaction with table-level limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.