ClickHouse / ClickHouse/ClickHouse

Provide atomic guarantees for EXCHANGE operation with multiple tables on Atomic DB Engine

Open
#37,783 4 comments 0 reactions 0 assignees View on GitHub
comp-database-engines external feature
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

Provide atomic guarantees for EXCHANGE operation with multiple tables on Atomic DB Engine
Currently [EXCHANGE operation](https://clickhouse.com/docs/en/sql-reference/statements/exchange) performs atomically by means of [renameat2 syscall](https://github.com/ClickHouse/ClickHouse/blob/090506a572c7334ffa10f8c04d756d387090656c/src/Common/atomicRename.cpp#L75)
Is't good enough for exchanging exactly two tables (`EXCHANGE a AND b`), but can cause non-atomicity in case of multiple tables (`EXCHANGE a AND b, c AND d`)

Exchange operation with multiple tables is mandatory for transaction-like modifications within Clickhouse Atomic database. Typical use case is following: create temporal copies of all tables with necessary mutations, atomic exchange original tables with modifies copies, and drop temporal copies
Currently only one step can cause problem with high-availability and fault-tolerance: atomic exchange with multiple tables. If database will crash in the middle of operation, database will run into inconsistent state.

Of course there are some methods to avoid data corruption, but all of them are dirty hacks, what you really don't want
Providing mutli-table atomicity in EXCHANGE operation is enough to prevent database corruption

Also, this feature is pretty simple to implement. Since every table has unique identifier, EXCHANGE operation can examine whenever all renames had been done successfully, and block database while renaming.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.