Add table configuration option that limits the number of segments a single query may hit
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
This will limit the amount of compute wasted on a single query, protecting against queries that bog down the cluster.
This is especially necessary in the case of https://github.com/apache/incubator-pinot/issues/6187 as materializing all of those segments on disc is expensive.
Contributor guide
Research direction
The issue names no files, tests, or entry points; start by locating table configuration handling and the query path that selects segments. Clarify the configuration semantics and how the limit interacts with queries like issue 6187. Done means a table-level setting prevents one query from hitting more than the configured number of segments, with tests covering the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100