apache / apache/pinot

[multistage][feature] allow configurable stage count limit to reject queries if they are too complex

Open
#9,369 0 comments 0 reactions 0 assignees View on GitHub
multi-stage
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

We should have the option reject SQL that are too complex (for example generating 10 stages execution plan) since they will simultaneously acquire execution resources and thus hogging up the entire cluster's resource.

This should be a cluster level configurable key for users to tune

Contributor guide

Open the contributing guide

Research direction

Start by tracing the multistage SQL query planning and execution path, then identify how cluster-level configurable keys are defined and read. Determine where the execution plan's stage count is available and how an over-limit query should be rejected; done means the limit is tunable at cluster level and complex queries are rejected before acquiring excessive execution resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.