ClickHouse / ClickHouse/ClickHouse
`ALTER TABLE ... DROP DETACHED PART/PARTITION` is not durable: the removal is never fsynced and no setting reaches this path, so the detached part resurrects after power loss
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
### Describe what's wrong
`ALTER TABLE t DROP DETACHED PARTITION ... SETTINGS allow_drop_detached = 1` acknowledges success, but the removal it performs — rename of `detached/` to `detached/deleting_` followed by a recursive unlink (`MergeTreeData::dropDetached` → `removeDetachedPart`) — issues no fsync and no directory sync. After a power loss within the filesystem-journal commit window (~5 s on default ext4), the directory operations roll back and the "dropped" detached part is back in `detached/`, fully intact, contradicting the acknowledgement.
Unlike `DROP PARTITION` on active parts (which is made durable by empty covering parts when `fsync_after_insert` + `fsync_part_directory` are on), **no setting reaches the detached-parts path**: `fsync_part_directory` is not consulted there. There is currently no way to make an acked `DROP DETACHED PART/PARTITION` durable.
Detached parts are exactly the state users purge for data-removal purposes — a detached part holds real data readable off disk — so a silently-resurrecting one matters.
### Does it reproduce on the most recent release?
Yes (reproduced on `26.7.1.1311`, current master nightly).
### How to reproduce
* Single node, data directory on ext4.
```sql
CREATE TABLE t (id UInt64, p UInt8) ENGINE = MergeTree ORDER BY id
SETTINGS fsync_after_insert = 1, fsync_part_directory = 1;
INSERT INTO t SELECT number, 1 FROM numbers(1000);
ALTER TABLE t DETACH PARTITION tuple();
-- sync(2) here: the detached part is durable on disk
ALTER TABLE t DROP DETACHED PARTITION tuple() SETTINGS allow_drop_detached = 1; -- acknowledged
-- power loss within the journal-commit window, restart
SELECT name FROM system.detached_parts WHERE table = 't'; -- all_1_1_0 (!)
```
Deterministic harness: data directory on ext4 over `dm-flakey` (power cut = suspend `--noflush` + `drop_writes` + remount; journal recovery yields the exact power-cut state). 4/4 deterministic with a wide journal-commit interval (`-o commit=60`); probabilistic within the `commit=` interval (default 5 s) otherwise.
Trust guards, all passing in every run:
* An un-fsynced canary file written just before the `DROP DETACHED` is always lost — the power cut is real.
* Graceful control: `DROP DETACHED` followed by a clean restart never resurrects the part — the resurrection is power-loss-specific, not restart logic.
### Expected behavior
Either the removal should be fsynced (sync the `detached/` directory after the unlinks, at least when the table has `fsync_part_directory = 1`), or the limitation should be documented.
### Error message and/or stacktrace
None — the resurrection is silent; the part simply reappears in `system.detached_parts`.
### Additional context
Sibling of #111348 (the un-fsynced Atomic-database DDL commit records: `DROP TABLE`/`RENAME`/`EXCHANGE` silently revert after power loss) and of #111322 (`DETACH ... PERMANENTLY` flag file never fsynced). Found by the same framework as #111269 / #111318 / #111320 / #111321 / #111330 / #111331 / #111339.
Contributor guide
Assessment
This issue has not been assessed yet.