ClickHouse / ClickHouse/ClickHouse

Workload Scheduling: Query slot flow control

Open
#84,983 0 comments 0 reactions 0 assignees View on GitHub
comp-scheduler feature
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
17h 34m
Merged PRs (30d)
479

Description

### Company or project name

_No response_

### Use case

At the moment, Query Slot Scheduling https://github.com/ClickHouse/ClickHouse/pull/78415 provides a few ways to limit the number of queries that are allowed to run (while other queries wait in a queue). Workload settings:
1. `max_queries_per_second` and `max_burst_queries` - throttling of a query flow.
2. `max_concurrent_queries` - simplest possible feedback loop, limiting the absolute number of concurrent queries in a workload.
3. `max_waiting_queries` https://github.com/ClickHouse/ClickHouse/pull/81250 - limits the number of waiting queries in a queue.

None of these ways enables controlling the latency of a query execution. And latency may be an essential SLI a user may want to control, especially for queries with expected subsecond latencies. There is no way to account for a suddenly occurring bottleneck that slows down query execution for any reason:
1. Throttling does not have feedback.
2. Limit on the absolute number of executing and waiting queries does not limit the response time.

### Describe the solution you'd like

There is no universal solution for this problem, so we have to explore possible implementation options. But the framework we should build on is an adaptive latency/throughput measurement and a dynamic limit on the number of queries computed as throughput-latency product.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Query Slot Scheduling in PR 78415 and the waiting-query limit in PR 81250. The issue calls for exploring an adaptive latency/throughput measurement framework and a dynamic query limit; done would require a decided implementation approach for controlling query execution latency.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.