ClickHouse / ClickHouse/ClickHouse

Flaky test `04640_query_log_map_columns_serialization`: `sum` over the prefetched read pool returns exactly 2^23 too much; `asynchronous_read_counters` sometimes empty in Fast test

Open
#113,762 3 comments 0 reactions 0 assignees View on GitHub
comp-mergetree flaky test
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

🕵 The test `04640_query_log_map_columns_serialization` fails about once a day in CI on unrelated PRs, in two distinct modes.

**Mode 1: `sum(k)` reads extra rows — always the same wrong value.** The first line of the reference (`SELECT sum(k) FROM t_query_log_maps` over `numbers(200000)`, expected `19999900000`) comes back as `20008288608` — exactly `2^23` more — on three different architectures:

```
@@ -1,5 +1,5 @@
-19999900000
+20008288608
```

- 2026-08-06, `Stateless tests (arm_binary, parallel)`, PR #112932
- 2026-08-05, `Stateless tests (arm_asan_ubsan, targeted)`, PR #109496
- 2026-08-04, `Stateless tests (amd_debug, parallel)`, PR #91062

The failing query pins `local_filesystem_read_method = 'pread_threadpool'`, `allow_prefetched_read_pool_for_local_filesystem = 1`, `filesystem_prefetch_step_marks = 1`, `merge_tree_min_rows_for_concurrent_read = 1`, `max_threads = 4` over a table with `index_granularity = 128`, so it exercises `MergeTreePrefetchedReadPool` with very fine-grained ranges. A too-large `sum` means some rows were read twice, which would be a correctness problem in the read path rather than in the test; the value being bit-exact reproducible (`+2^23`) across architectures suggests a deterministic range-splitting corner rather than a data race. The automatic diagnosis re-ran the test 112 times with the same randomized settings and could not reproduce.

**Mode 2 (Fast test only): `asynchronous_read_counters` is empty.**

```
-['max_parallel_prefetch_tasks','total_prefetch_tasks']
+[]
```

- 2026-08-05, `Fast test (arm_darwin)`, PR #112945
- 2026-07-28, `Fast test`, PR #103706

The test's comment already lists the conditions under which the prefetched read pool runs; apparently there is a rare scheduling case (or a Fast-test-specific one) where no prefetch task is ever created and the map stays empty.

CIDB history: https://play.clickhouse.com/play?user=play#U0VMRUNUIGNoZWNrX3N0YXJ0X3RpbWUsIGNoZWNrX25hbWUsIHB1bGxfcmVxdWVzdF9udW1iZXIsIHJlcG9ydF91cmwgRlJPTSBjaGVja3MgV0hFUkUgdGVzdF9uYW1lID0gJzA0NjQwX3F1ZXJ5X2xvZ19tYXBfY29sdW1uc19zZXJpYWxpemF0aW9uJyBBTkQgdGVzdF9zdGF0dXMgSU4gKCdGQUlMJywgJ0VSUk9SJykgT1JERVIgQlkgY2hlY2tfc3RhcnRfdGltZSBERVNDIExJTUlUIDIw

Contributor guide

Open the contributing guide

Research direction

Start by running the targeted test `04640_query_log_map_columns_serialization` under the listed prefetch settings and inspect `MergeTreePrefetchedReadPool` and its range-splitting path. Determine whether the test can produce duplicate rows or skip prefetch-task counter initialization; done means the aggregate is correct and `asynchronous_read_counters` is populated reliably in both Fast and CI modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.