ClickHouse / ClickHouse/ClickHouse

Approximate GROUP BY with ORDER BY ... LIMIT distributed query.

Open
#20,587 0 comments 0 reactions 0 assignees View on GitHub
comp-distributed performance
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

When doing distributed query with

GROUP BY ... ORDER BY ... LIMIT ...

we can get approximate result if perform ORDER BY and LIMIT on every shard and then merge filtered data on the initiating node.

It will send less data over network making query tremendously faster.
The results can be accurate enough and it will be useful to make this behaviour available under a setting.

The result will be accurate if top keys appeared to be the same on every shard.
We can also allow to tune the accuracy by multiplying limit to some value.

See also:
\- `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`;

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.