ClickHouse / ClickHouse/ClickHouse
Allow to attach a part/partition from a table with more granular partitioning.
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 17h 34m
- Merged PRs (30d)
- 479
Description
if there are 2 table which differs only in the partitioning key and the part from source table matches single partition in target table - allow that (may be with setting like 'allow_attaching_compatible_partitioning_key_parts`).
```
create table source engine=MergeTree ORDER BY tuple() PARTITION BY toYYYYMMDD(timestamp) AS SELECT number, toDateTime('2010-01-01 00:00:00') as timestamp from numbers(100);
create table target engine=MergeTree ORDER BY tuple() PARTITION BY toYYYYMM(timestamp) AS SELECT number, toDateTime('2010-01-01 00:00:00') as timestamp from numbers(100);
ALTER TABLE target ATTACH PARTITION ID '20100101' FROM source;
```
```
Code: 36, e.displayText() = DB::Exception: Tables have different partition key (version 20.7.1.4186 (official build)) (from 127.0.0.1:39986) (in query: ALTER TABLE target ATTACH PARTITION ID '20100101' FROM source;), Stack trace (when copying this message, always include the lines below):
0. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0x123b04c0 in /usr/lib/debug/usr/bin/clickhouse
1. /build/obj-x86_64-linux-gnu/../src/Common/Exception.cpp:37: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xa2a02bd in /usr/lib/debug/usr/bin/clickhouse
2. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/string:2134: DB::MergeTreeData::checkStructureAndGetMergeTreeData(DB::IStorage&, std::__1::shared_ptr const&, std::__1::shared_ptr const&) const (.cold) @ 0xf8a32f9 in /usr/lib/debug/usr/bin/clickhouse
3. /build/obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1146: DB::StorageMergeTree::replacePartitionFrom(std::__1::shared_ptr const&, std::__1::shared_ptr const&, bool, DB::Context const&) @ 0xf6a7100 in /usr/lib/debug/usr/bin/clickhouse
4. /build/obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1057: DB::StorageMergeTree::alterPartition(std::__1::shared_ptr const&, std::__1::shared_ptr const&, std::__1::vector > const&, DB::Context const&) @ 0xf6aa4bf in /usr/lib/debug/usr/bin/clickhouse
5. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:662: DB::InterpreterAlterQuery::execute() @ 0xf169433 in /usr/lib/debug/usr/bin/clickhouse
6. /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:386: DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) @ 0xf4d1bc2 in /usr/lib/debug/usr/bin/clickhouse
7. /build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:655: DB::executeQuery(std::__1::basic_string, std::__1::allocator > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) @ 0xf4d5405 in /usr/lib/debug/usr/bin/clickhouse
8. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:253: DB::TCPHandler::runImpl() @ 0xfb39ba3 in /usr/lib/debug/usr/bin/clickhouse
9. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1215: DB::TCPHandler::run() @ 0xfb3a99e in /usr/lib/debug/usr/bin/clickhouse
10. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x122ce2eb in /usr/lib/debug/usr/bin/clickhouse
11. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0x122ce77b in /usr/lib/debug/usr/bin/clickhouse
12. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0x1244d2a6 in /usr/lib/debug/usr/bin/clickhouse
13. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223: Poco::ThreadImpl::runnableEntry(void*) @ 0x124486a0 in /usr/lib/debug/usr/bin/clickhouse
14. start_thread @ 0x9609 in /lib/x86_64-linux-gnu/libpthread-2.31.so
15. /build/glibc-YYA7BZ/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97: __clone @ 0x122103 in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.31.so
Received exception from server (version 20.7.1):
Code: 36. DB::Exception: Received from localhost:9000. DB::Exception: Tables have different partition key.
```
Internally what should happen:
1) we copy (by hardlinks) the part (as now)
2) we recalculate min/max for the column used in partitioning key if it ends up with different partitions - fail with exception, if it's a single partitions - write new minmax file (see IMergeTreeDataPart::MinMaxIndex)
3) recalc checksum file
4) do rest as usual
Contributor guide
Assessment
This issue has not been assessed yet.