ClickHouse / ClickHouse/ClickHouse

Cannot remove outdated data parts

Open
#96,037 0 comments 0 reactions 0 assignees View on GitHub
comp-mergetree external potential bug
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Company or project name

_No response_

### Describe what's wrong

In the same shard, there are two replicas, Replica 01 and Replica 02.

Replica 01 inserts a data part named 20260205_0_0_0. Before this part becomes active, Replica 02 has already finished fetching and merging corresponding parts.

When Replica 01 detects that its newly inserted part is fully covered by the merged part on Replica 02, it marks the part(20260205_0_0_0) as Outdated.

However, this outdated part cannot be removed and remains on disk, with the following log message:

"Part maybe visible for transactions"

```
SELECT
partition,
name,
active,
rows,
modification_time,
removal_state,
delete_ttl_info_min,
delete_ttl_info_max,
last_removal_attempt_time
FROM system.parts
WHERE `table` = 'merge_tree_test_remove'
ORDER BY active ASC

┌─partition─┬─name───────────┬─active─┬─rows─┬───modification_time─┬─removal_state────────────────────────────┬─delete_ttl_info_min─┬─delete_ttl_info_max─┬─last_removal_attempt_time─┐
1. │ 20260205 │ 20260205_0_0_0 │ 0 │ 1 │ 2026-02-05 16:47:41 │ Part maybe visible for transactions │ 1970-01-01 08:00:00 │ 1970-01-01 08:00:00 │ 2026-02-05 16:58:03 │
2. │ 20260205 │ 20260205_0_1_1 │ 1 │ 2 │ 2026-02-05 16:47:50 │ Cleanup thread hasn't seen this part yet │ 1970-01-01 08:00:00 │ 1970-01-01 08:00:00 │ 1970-01-01 08:00:00 │
└───────────┴────────────────┴────────┴──────┴─────────────────────┴──────────────────────────────────────────┴─────────────────────┴─────────────────────┴───────────────────────────┘

```

### Does it reproduce on the most recent release?

Yes

### How to reproduce

version: 26.2.1.1

The issue can be reproduced by introducing a sleep before acquiring the data parts lock.

### Expected behavior

_No response_

### Error message and/or stacktrace

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.