ClickHouse / ClickHouse/ClickHouse

Docs: document merge selection algorithm

Open
#112,125 0 comments 0 reactions 0 assignees View on GitHub
comp-documentation
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

Our user docs should describe at a high level how the default merge selector algorithm in ClickHouse work (i.e. which parts are selected for merge based on what criteria). The description would fit [here](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree) or [here](https://clickhouse.com/docs/en/sql-reference/statements/optimize). We should note that that thresholds and details of the algorithms are not part of the stable database interface, the docs would only apply to the latest version and they may change between versions (this already applies to system tables, for example).

The current heuristics are described
- [here](https://github.com/ClickHouse/ClickHouse/issues/16595#issuecomment-720642118) (I was told the logic did not change significantly since 2020), and
- [here](https://github.com/ClickHouse/ClickHouse/blob/02b564995e9ff42dea6897d03504a4411139ca2a/src/Storages/MergeTree/Compaction/MergeSelectors/SimpleMergeSelector.h) (as source code comments). The heuristics were made exchangeable recently and there are [more of them](https://github.com/ClickHouse/ClickHouse/tree/02b564995e9ff42dea6897d03504a4411139ca2a/src/Storages/MergeTree/Compaction/MergeSelectors) but SimpleMergeSelector is the default one.

In case of questions, @Michicosun can help.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.