ClickHouse / ClickHouse/ClickHouse
Provide atomic guarantees for EXCHANGE operation with multiple tables on Atomic DB Engine
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
Provide atomic guarantees for EXCHANGE operation with multiple tables on Atomic DB Engine
Currently [EXCHANGE operation](https://clickhouse.com/docs/en/sql-reference/statements/exchange) performs atomically by means of [renameat2 syscall](https://github.com/ClickHouse/ClickHouse/blob/090506a572c7334ffa10f8c04d756d387090656c/src/Common/atomicRename.cpp#L75)
Is't good enough for exchanging exactly two tables (`EXCHANGE a AND b`), but can cause non-atomicity in case of multiple tables (`EXCHANGE a AND b, c AND d`)
Exchange operation with multiple tables is mandatory for transaction-like modifications within Clickhouse Atomic database. Typical use case is following: create temporal copies of all tables with necessary mutations, atomic exchange original tables with modifies copies, and drop temporal copies
Currently only one step can cause problem with high-availability and fault-tolerance: atomic exchange with multiple tables. If database will crash in the middle of operation, database will run into inconsistent state.
Of course there are some methods to avoid data corruption, but all of them are dirty hacks, what you really don't want
Providing mutli-table atomicity in EXCHANGE operation is enough to prevent database corruption
Also, this feature is pretty simple to implement. Since every table has unique identifier, EXCHANGE operation can examine whenever all renames had been done successfully, and block database while renaming.
Contributor guide
Assessment
This issue has not been assessed yet.