ClickHouse / ClickHouse/ClickHouse

replication queue scheduling strategy

Open
#77,289 1 comment 0 reactions 0 assignees View on GitHub
comp-replication external question
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Company or project name

_No response_

### Question

I have a clickhouse cluster with 1 shard, 2 replicas, all tables are ReplicatedReplacingMergeTree engine. I add a new replica node into the cluster. On the new node I create tables one by one, when one table's replication queue is 0, the next table will be created.
When replicating a large table A, the replication queue increases rapidly. background_fetches_pool_size is set to 16, and replication_queue shows that there are 16 GET_PART queues for table A. The waiting queues for the other tables have been growing continuously, and the total number of queues reaches tens of thousands.
What is the scheduling strategy for replication queues please? Is there any way to prevent other table replicas from being polled because of a single large table replication?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.