ClickHouse / ClickHouse/ClickHouse

Large replication queue caused by no-op mutation on old data

Open
#90,542 3 comments 2 reactions 0 assignees View on GitHub
comp-mutations external question
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
20h 33m
Merged PRs (30d)
501

Description

### Company or project name

_No response_

### Question

We created an index without materializing it on a large table with a significant amount of historical data in cloud storage.

Afterwards, we dropped the index via `ALTER TABLE DROP INDEX` since we thought it would not mutate the old parts. The subsequent mutation still "modified" those old parts, as we can see `modification_time` in system.parts has been updated.

This triggered a very large amount of `GET_PART` entries in the replication queue on the other replica for this shard. Enough that it will take a significant amount of time and bandwidth to perform the full pull/replicate loop from cloud storage.

Since the actual data in any of these parts has not changed, is there a way we can effectively drop the replication queue, or trick the replica into thinking it is up to date?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.