ClickHouse / ClickHouse/ClickHouse

Iceberg out-of-bounds with a materialized view

Open
#109,835 3 comments 0 reactions 0 assignees View on GitHub
bug comp-datalake crash fuzz
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Describe the bug

Easy to reproduce.

### How to reproduce

Start the server, then run [repro.sh](https://github.com/user-attachments/files/29813448/repro.sh)

### Error message and/or stacktrace

Stack trace:
```
contrib/llvm-project/libcxx/include/__vector/vector.h:417: libc++ Hardening assertion __n < size() failed: vector[] index out of bounds
2026.07.08 18:39:15.480246 [ 1018154 ] {} BaseDaemon: Received signal 6
2026.07.08 18:39:15.480293 [ 1018154 ] {} BaseDaemon: ########## Short fault info ############
2026.07.08 18:39:15.480308 [ 1018154 ] {} BaseDaemon: (version 26.7.1.644 (official build), build id: D26EF1B62A75E5DBD5DAF8FC9BB79AC24DD85E3F, git hash: 0633b44a6338d249ec3ce7a9a851aac58f6be2d7, architecture: x86_64) (from thread 1018462) Received signal 6 (internal)
2026.07.08 18:39:15.480311 [ 1018154 ] {} BaseDaemon: Signal description: Aborted
2026.07.08 18:39:15.480314 [ 1018154 ] {} BaseDaemon: Sent by tkill.
2026.07.08 18:39:15.480325 [ 1018154 ] {} BaseDaemon: Stack trace: 0x0000713bcb2a648d 0x0000713bcb245b7e 0x0000713bcb2288ec 0x000060425d6e4265 0x00006042485c3168 0x00006042485eecc6 0x00006042485fd449 0x00006042485fc908 0x00006042485fc82b 0x0000604251140606 0x0000604251140206 0x0000604250e722bd 0x0000604250e64fbc 0x0000604250e6576c 0x0000604250e686e3 0x00006042446c5049 0x00006042446ce04b 0x00006042446c227c 0x00006042446cb40e 0x0000713bcb2a40da 0x0000713bcb3377ac
2026.07.08 18:39:15.480329 [ 1018154 ] {} BaseDaemon: ########################################
2026.07.08 18:39:15.480334 [ 1018154 ] {} BaseDaemon: (version 26.7.1.644 (official build), build id: D26EF1B62A75E5DBD5DAF8FC9BB79AC24DD85E3F, git hash: 0633b44a6338d249ec3ce7a9a851aac58f6be2d7) (from thread 1018462) (query_id: 730891b9-a651-41e8-91b1-37f95471a5df) (query: INSERT INTO default.src (c0, c1, c2) FORMAT Values) Received signal Aborted (6)
2026.07.08 18:39:15.480336 [ 1018154 ] {} BaseDaemon: Sent by tkill.
2026.07.08 18:39:15.480340 [ 1018154 ] {} BaseDaemon: Stack trace: 0x0000713bcb2a648d 0x0000713bcb245b7e 0x0000713bcb2288ec 0x000060425d6e4265 0x00006042485c3168 0x00006042485eecc6 0x00006042485fd449 0x00006042485fc908 0x00006042485fc82b 0x0000604251140606 0x0000604251140206 0x0000604250e722bd 0x0000604250e64fbc 0x0000604250e6576c 0x0000604250e686e3 0x00006042446c5049 0x00006042446ce04b 0x00006042446c227c 0x00006042446cb40e 0x0000713bcb2a40da 0x0000713bcb3377ac
2026.07.08 18:39:15.480373 [ 1018154 ] {} BaseDaemon: 3. pthread_kill @ 0x00000000000a648d
2026.07.08 18:39:15.480386 [ 1018154 ] {} BaseDaemon: 4. raise @ 0x0000000000045b7e
2026.07.08 18:39:15.480394 [ 1018154 ] {} BaseDaemon: 5. abort @ 0x00000000000288ec
2026.07.08 18:39:15.488511 [ 1018154 ] {} BaseDaemon: 6. contrib/llvm-project/libcxx/src/verbose_abort.cpp:62:3: ? @ 0x000000002e698265
2026.07.08 18:39:15.498528 [ 1018154 ] {} BaseDaemon: 7.0. inlined from contrib/llvm-project/libcxx/include/__vector/vector.h:417: std::vector>::operator[][abi:sqe220101](unsigned long) const
2026.07.08 18:39:15.498535 [ 1018154 ] {} BaseDaemon: 7. src/Storages/ObjectStorage/DataLakes/Iceberg/DataFileStatistics.cpp:97:5: DB::DataFileStatistics::getColumnSizes() const @ 0x0000000019577168
2026.07.08 18:39:15.559850 [ 1018154 ] {} BaseDaemon: 8. src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp:363:53: DB::generateManifestFile(Poco::SharedPtr>, std::vector> const&, std::vector> const&, std::vector, AllocatorWithMemoryTracking>> const&, std::vector> const&, std::vector> const&, std::vector> const&, std::optional const&, std::shared_ptr, Poco::SharedPtr>, String const&, Poco::SharedPtr>, long, DB::WriteBuffer&, DB::Iceberg::FileContentType, std::optional) @ 0x00000000195a2cc6
2026.07.08 18:39:15.605583 [ 1018154 ] {} BaseDaemon: 9. src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp:1023:17: DB::IcebergStorageSink::initializeMetadata() @ 0x00000000195b1449
2026.07.08 18:39:15.650481 [ 1018154 ] {} BaseDaemon: 10. src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp:879:13: DB::IcebergStorageSink::finalizeBuffers() @ 0x00000000195b0908
2026.07.08 18:39:15.695457 [ 1018154 ] {} BaseDaemon: 11. src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp:854:5: DB::IcebergStorageSink::onFinish() @ 0x00000000195b082b
2026.07.08 18:39:15.705017 [ 1018154 ] {} BaseDaemon: 12.0. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:502: ?
2026.07.08 18:39:15.705026 [ 1018154 ] {} BaseDaemon: 12.1. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:754: ?
2026.07.08 18:39:15.705030 [ 1018154 ] {} BaseDaemon: 12. src/Processors/Transforms/ExceptionKeepingTransform.cpp:106:12: DB::runStep(std::function, std::shared_ptr&) @ 0x00000000220f4606
2026.07.08 18:39:15.710794 [ 1018154 ] {} BaseDaemon: 13. src/Processors/Transforms/ExceptionKeepingTransform.cpp:189:42: DB::ExceptionKeepingTransform::work() @ 0x00000000220f4206
2026.07.08 18:39:15.715105 [ 1018154 ] {} BaseDaemon: 14.0. inlined from src/Processors/Executors/ExecutionThreadContext.cpp:54: DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*)
2026.07.08 18:39:15.715121 [ 1018154 ] {} BaseDaemon: 14. src/Processors/Executors/ExecutionThreadContext.cpp:103:28: DB::ExecutionThreadContext::executeTask() @ 0x0000000021e262bd
2026.07.08 18:39:15.729289 [ 1018154 ] {} BaseDaemon: 15. src/Processors/Executors/PipelineExecutor.cpp:384:26: DB::PipelineExecutor::executeStepImpl(unsigned long, DB::WorkloadResources&&, std::atomic*) @ 0x0000000021e18fbc
2026.07.08 18:39:15.742202 [ 1018154 ] {} BaseDaemon: 16. src/Processors/Executors/PipelineExecutor.cpp:352:5: DB::PipelineExecutor::executeSingleThread(unsigned long, DB::WorkloadResources&&) @ 0x0000000021e1976c
2026.07.08 18:39:15.757342 [ 1018154 ] {} BaseDaemon: 17.0. inlined from src/Processors/Executors/PipelineExecutor.cpp:689: operator()
2026.07.08 18:39:15.757355 [ 1018154 ] {} BaseDaemon: 17.1. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:90: std::__invoke_result_impl)::$_0&>::type std::__invoke[abi:sqe220101])::$_0&>(DB::PipelineExecutor::spawnThreads(std::shared_ptr)::$_0&)
2026.07.08 18:39:15.757363 [ 1018154 ] {} BaseDaemon: 17.2. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:350: void std::__invoke_void_return_wrapper::__call[abi:sqe220101])::$_0&>(DB::PipelineExecutor::spawnThreads(std::shared_ptr)::$_0&)
2026.07.08 18:39:15.757371 [ 1018154 ] {} BaseDaemon: 17.3. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:356: void std::__invoke_r[abi:sqe220101])::$_0&>(DB::PipelineExecutor::spawnThreads(std::shared_ptr)::$_0&)
2026.07.08 18:39:15.757374 [ 1018154 ] {} BaseDaemon: 17. contrib/llvm-project/libcxx/include/__functional/function.h:443:17: ? @ 0x0000000021e1c6e3
2026.07.08 18:39:15.767760 [ 1018154 ] {} BaseDaemon: 18.0. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:502: ?
2026.07.08 18:39:15.767768 [ 1018154 ] {} BaseDaemon: 18.1. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:754: ?
2026.07.08 18:39:15.767771 [ 1018154 ] {} BaseDaemon: 18. src/Common/ThreadPool.cpp:1103:12: ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x0000000015679049
2026.07.08 18:39:15.786520 [ 1018154 ] {} BaseDaemon: 19.0. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:502: ?
2026.07.08 18:39:15.786525 [ 1018154 ] {} BaseDaemon: 19.1. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:754: ?
2026.07.08 18:39:15.786528 [ 1018154 ] {} BaseDaemon: 19.2. inlined from src/Common/ThreadPool.cpp:1293: operator()
2026.07.08 18:39:15.786537 [ 1018154 ] {} BaseDaemon: 19.3. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:90: std::__invoke_result_impl, std::function, unsigned long, unsigned long, bool, bool)::$_0&>::type std::__invoke[abi:sqe220101], std::function, unsigned long, unsigned long, bool, bool)::$_0&>(startThreadFromGlobalPool(std::shared_ptr, std::function, unsigned long, unsigned long, bool, bool)::$_0&)
2026.07.08 18:39:15.786545 [ 1018154 ] {} BaseDaemon: 19.4. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:350: void std::__invoke_void_return_wrapper::__call[abi:sqe220101], std::function, unsigned long, unsigned long, bool, bool)::$_0&>(startThreadFromGlobalPool(std::shared_ptr, std::function, unsigned long, unsigned long, bool, bool)::$_0&)
2026.07.08 18:39:15.786550 [ 1018154 ] {} BaseDaemon: 19.5. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:356: void std::__invoke_r[abi:sqe220101], std::function, unsigned long, unsigned long, bool, bool)::$_0&>(startThreadFromGlobalPool(std::shared_ptr, std::function, unsigned long, unsigned long, bool, bool)::$_0&)
2026.07.08 18:39:15.786552 [ 1018154 ] {} BaseDaemon: 19. contrib/llvm-project/libcxx/include/__functional/function.h:443:12: ? @ 0x000000001568204b
2026.07.08 18:39:15.796127 [ 1018154 ] {} BaseDaemon: 20.0. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:502: ?
2026.07.08 18:39:15.796135 [ 1018154 ] {} BaseDaemon: 20.1. inlined from contrib/llvm-project/libcxx/include/__functional/function.h:754: ?
2026.07.08 18:39:15.796138 [ 1018154 ] {} BaseDaemon: 20. src/Common/ThreadPool.cpp:1113:12: ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000001567627c
2026.07.08 18:39:15.814578 [ 1018154 ] {} BaseDaemon: 21.0. inlined from contrib/llvm-project/libcxx/include/__type_traits/invoke.h:0: std::__invoke_result_impl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>::type std::__invoke[abi:sqe220101]::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>(void (ThreadPoolImpl::ThreadFromThreadPool::*&&)(), ThreadPoolImpl::ThreadFromThreadPool*&&)
2026.07.08 18:39:15.814592 [ 1018154 ] {} BaseDaemon: 21.1. inlined from contrib/llvm-project/libcxx/include/__thread/thread.h:161: void std::__thread_execute[abi:sqe220101]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*, 0ul, 1ul>(std::tuple>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>&, std::__integer_sequence)
2026.07.08 18:39:15.814597 [ 1018154 ] {} BaseDaemon: 21. contrib/llvm-project/libcxx/include/__thread/thread.h:169: void* std::__thread_proxy[abi:sqe220101]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000001567f40e
2026.07.08 18:39:15.814609 [ 1018154 ] {} BaseDaemon: 22. __sin_avx @ 0x00000000000a40da
2026.07.08 18:39:15.814614 [ 1018154 ] {} BaseDaemon: 23. __clone3 @ 0x00000000001377ac
2026.07.08 18:39:16.059120 [ 1018154 ] {} BaseDaemon: Integrity check of the executable successfully passed (checksum: 557C964CAD61BF92FEC8D7ECCB790292)
2026.07.08 18:39:16.059158 [ 1018154 ] {} BaseDaemon: ClickHouse version 26.7.1.644 is old and should be upgraded to the latest version.
2026.07.08 18:39:16.059217 [ 1018154 ] {} BaseDaemon: Changed settings: s3_truncate_on_insert = true, azure_truncate_on_insert = true, allow_suspicious_low_cardinality_types = true, allow_suspicious_fixed_string_types = true, allow_suspicious_indices = true, allow_suspicious_ttl_expressions = true, allow_suspicious_variant_types = true, allow_suspicious_primary_key = true, allow_suspicious_types_in_group_by = true, allow_suspicious_types_in_order_by = true, enable_positional_arguments = true, allow_nonconst_timezone_arguments = true, enable_time_time64_type = true, allow_nondeterministic_optimize_skip_unused_shards = true, table_function_remote_max_addresses = 200, insert_allow_materialized_columns = true, allow_distributed_ddl = true, allow_suspicious_codecs = true, max_execution_time = 60., allow_custom_error_code_in_throwif = true, allow_ddl = true, allow_introspection_functions = true, allow_drop_detached = true, allow_experimental_url_wildcard_from_index_pages = true, enable_lightweight_delete = true, allow_materialized_view_with_bad_select = true, allow_settings_after_format_in_insert = true, allow_nondeterministic_mutations = true, allow_non_metadata_alters = true, enable_materialized_cte = true, enable_lightweight_update = true, query_cache_nondeterministic_function_handling = 'ignore', query_cache_system_table_handling = 'ignore', allow_experimental_kafka_offsets_storage_in_keeper = true, engine_file_truncate_on_insert = true, database_replicated_allow_replicated_engine_arguments = 1, allow_experimental_correlated_subqueries = true, keeper_max_retries = 15, enable_order_by_all = true, allow_dynamic_type_in_join_keys = true, allow_create_index_without_type = true, allow_named_collection_override_by_default = true, allow_experimental_geo_types_in_iceberg = true, allow_experimental_delta_lake_writes = true, parallel_replicas_for_non_replicated_merge_tree = true, parallel_replicas_allow_materialized_views = true, allow_experimental_database_iceberg = true, allow_experimental_database_unity_catalog = true, allow_experimental_database_glue_catalog = true, allow_experimental_analyzer = true, allow_experimental_nullable_tuple_type = true, allow_experimental_materialized_postgresql_table = true, allow_experimental_funnel_functions = true, allow_experimental_nlp_functions = true, allow_experimental_hash_functions = true, allow_experimental_time_series_table = true, allow_experimental_unique_key = true, allow_experimental_codecs = true, allow_experimental_join_right_table_sorting = true, allow_experimental_json_lazy_type_hints = true, enable_streaming_queries = true, enable_full_text_index = true, allow_experimental_window_view = true, allow_experimental_database_materialized_postgresql = true, allow_nullable_tuple_in_extracted_subcolumns = true, allow_experimental_database_hms_catalog = true, allow_experimental_kusto_dialect = true, allow_experimental_prql_dialect = true, allow_experimental_polyglot_dialect = true, allow_experimental_delta_kernel_rs = true, allow_insert_into_iceberg = true, allow_experimental_cleanup_old_data_files_compaction = true, allow_experimental_iceberg_compaction = true, allow_iceberg_remove_orphan_files = true, allow_experimental_expire_snapshots = true, allow_experimental_ytsaurus_table_engine = true, allow_experimental_ytsaurus_table_function = true, allow_experimental_ytsaurus_dictionary_source = true, allow_experimental_time_series_aggregate_functions = true, allow_experimental_paimon_storage_engine = true, allow_experimental_object_storage_queue_hive_partitioning = true, allow_experimental_database_paimon_rest_catalog = true, allow_experimental_ai_functions = true, allow_experimental_query_deduplication = true, allow_experimental_alias_table_engine = true, allow_not_comparable_types_in_order_by = true, allow_not_comparable_types_in_comparison_functions = true, enable_zstd_qat_codec = true, enable_deflate_qpl_codec = true, input_format_parallel_parsing = false, type_json_skip_duplicated_paths = true, input_format_parquet_allow_geoparquet_parser = true
```

Contributor guide

Open the contributing guide

Research direction

Run the linked repro.sh against the server and inspect the failure at src/Storages/ObjectStorage/DataLakes/Iceberg/DataFileStatistics.cpp:97. Read the surrounding manifest-generation and write flow in IcebergWrites.cpp, especially generateManifestFile and IcebergStorageSink::initializeMetadata; done means the materialized-view INSERT no longer aborts with the out-of-bounds assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.