ClickHouse / ClickHouse/ClickHouse
Problems with tables with null bytes in a column name
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Describe the unexpected behaviour**
Attach and detach of parts of tables with null byte in their name doesn't work.
**How to reproduce**
(In python)
```
import clickhouse_connect
client = clickhouse_connect.get_client(host='localhost')
client.command('''
CREATE TABLE example_table
(
u1 UInt64
) ENGINE = MergeTree()
PRIMARY KEY u1;
'''
)
client.command("ALTER TABLE example_table ADD COLUMN `u\x002` UInt64;")
client.command("insert into example_table Values (0, 0);")
client.command("ALTER TABLE example_table DETACH PART 'all_1_1_0';")
client.command("ALTER TABLE example_table ATTACH PART 'all_1_1_0';")
```
**Expected behavior**
Normal DETACH and ATTACH.
**Error message and/or stacktrace**
```
2024.09.06 13:43:12.924050 [ 847121 ] {a0aeaefe-7164-4b7c-9ad0-450c0ede6835} DynamicQueryHandler: Code: 246. DB::Exception: Found unexpected column 'u' in serialization infos: Part contains files: serialization.json (151 bytes), count.txt (1 bytes), default_compression_codec.txt (10 bytes), metadata_version.txt (1 bytes), primary.cidx (42 bytes), checksums.txt (259 bytes), columns.txt (63 bytes), data.cmrk3 (58 bytes), data.bin (70 bytes): Part is empty. (CORRUPTED_DATA), Stack trace (when copying this message, always include the lines below):
0. /home/ubuntu/ClickHouseDev2/ClickHouse/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x000000003700a7f1
1. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Common/Exception.cpp:109: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001e3d17ed
2. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Common/Exception.h:110: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000e5db4c6
3. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Common/Exception.h:128: DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x000000000e6448f5
4. /home/ubuntu/ClickHouseDev2/ClickHouse/src/DataTypes/Serializations/SerializationInfo.cpp:291: DB::SerializationInfoByName::readJSON(DB::NamesAndTypesList const&, DB::SerializationInfoSettings const&, DB::ReadBuffer&) @ 0x000000002a7a74cc
5. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Storages/MergeTree/IMergeTreeDataPart.cpp:1635: DB::IMergeTreeDataPart::loadColumns(bool) @ 0x000000002e806b22
6. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Storages/MergeTree/IMergeTreeDataPart.cpp:717: DB::IMergeTreeDataPart::loadColumnsChecksumsIndexes(bool, bool) @ 0x000000002e8036a9
7. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Storages/MergeTree/MergeTreeData.cpp:4905: DB::MergeTreeData::tryLoadPartsToAttach(std::shared_ptr const&, bool, std::shared_ptr, DB::MergeTreeData::PartsTemporaryRename&) @ 0x000000002ea0fb1d
8. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Storages/StorageMergeTree.cpp:2051: DB::StorageMergeTree::attachPartition(std::shared_ptr const&, std::shared_ptr const&, bool, std::shared_ptr) @ 0x000000002f03b8ba
9. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Storages/MergeTree/MergeTreeData.cpp:5311: DB::MergeTreeData::alterPartition(std::shared_ptr const&, std::vector> const&, std::shared_ptr) @ 0x000000002e9f6524
10. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Interpreters/InterpreterAlterQuery.cpp:235: DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x000000002bcef55e
11. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Interpreters/InterpreterAlterQuery.cpp:70: DB::InterpreterAlterQuery::execute() @ 0x000000002bcebb31
12. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Interpreters/executeQuery.cpp:1236: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x000000002c5c643a
13. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Interpreters/executeQuery.cpp:1534: DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::shared_ptr, std::function, DB::QueryFlags, std::optional const&, std::function const&, std::optional const&)>) @ 0x000000002c5cd3a4
14. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Server/HTTPHandler.cpp:529: DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::optional&, StrongTypedef const&) @ 0x000000002f430645
15. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Server/HTTPHandler.cpp:705: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&, StrongTypedef const&) @ 0x000000002f43643b
16. /home/ubuntu/ClickHouseDev2/ClickHouse/src/Server/HTTP/HTTPServerConnection.cpp:71: DB::HTTPServerConnection::run() @ 0x000000002f568205
17. /home/ubuntu/ClickHouseDev2/ClickHouse/base/poco/Net/src/TCPServerConnection.cpp:43: Poco::Net::TCPServerConnection::start() @ 0x00000000371c708f
18. /home/ubuntu/ClickHouseDev2/ClickHouse/base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x00000000371c7c97
19. /home/ubuntu/ClickHouseDev2/ClickHouse/base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x00000000370e957c
20. /home/ubuntu/ClickHouseDev2/ClickHouse/base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000370e4228
21. asan_thread_start(void*) @ 0x000000000e591059
22. ? @ 0x0000765838e94ac3
23. ? @ 0x0000765838f26850
```
**Additional context**
The problem is more or less clear. During serialization.json parsing the problem with zero byte occurs. That's why the columns in table and columns parsed from the json file are parsed in a different way.
I think that the problem is here: [https://github.com/ClickHouse/ClickHouse/blob/master/src/DataTypes/Serializations/SerializationInfo.cpp#L282](url) (String is not parsed fully because const char* contains internal null bytes)
Contributor guide
Assessment
This issue has not been assessed yet.