ClickHouse / ClickHouse/ClickHouse
Nested LowCardinality does not engage low_cardinality_key_string for aggregations
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
### Company or project name
_No response_
### Describe the unexpected behaviour
I have a table that as a bunch of columns and then a mapping from stacks to their counts to make flamegraphs. Stripping everything else, it can be expressed in two ways:
1. As a flat table (not ideal if you have many other columns that would need duplication):
```
:) create table flat_stacks (stack LowCardinality(String), value UInt64 CODEC(ZSTD(1))) engine = MergeTree
```
2. As a nested column:
```
:) create table nested_stacks (stackMap Nested(stack LowCardinality(String), value UInt64) CODEC(ZSTD(1))) engine = MergeTree
```
Let's insert the same data into both:
```
:) insert into flat_stacks select stackMap.stack as stack, stackMap.value as value from welp array join stackMap
:) insert into nested_stacks select stackMap.stack, stackMap.value from welp
```
Here's how it looks like in terms of data on disk:
```
┌─table─────────┬─column─────────┬────rows─┬─uncompressed─┬─compressed─┬─uncompressed_per_row─┬─compressed_per_row─┬─compression_ratio─┐
1. │ flat_stacks │ stack │ 9593661 │ 7661036261 │ 381306586 │ 798.55 │ 39.74 │ 20.09 │
2. │ flat_stacks │ value │ 9593661 │ 76749288 │ 8161820 │ 8.00 │ 0.85 │ 9.40 │
3. │ nested_stacks │ stackMap.stack │ 3646 │ 7345456902 │ 236466574 │ 2014661.79 │ 64856.43 │ 31.06 │
4. │ nested_stacks │ stackMap.value │ 3646 │ 76749288 │ 8042077 │ 21050.27 │ 2205.72 │ 9.54 │
└───────────────┴────────────────┴─────────┴──────────────┴────────────┴──────────────────────┴────────────────────┴───────────────────┘
```
Now let's query the flat table:
```
:) select count() from (select stack, sum(value) from flat_stacks group by stack)
...
[cube] 2025.12.08 04:48:48.202084 [ 803 ] {a3a04416-b2a4-4ccb-8297-62904397364b} Aggregator: Aggregation method: low_cardinality_key_string_two_level
...
[cube] 2025.12.08 04:48:58.935135 [ 3725 ] {a3a04416-b2a4-4ccb-8297-62904397364b} executeQuery: Read 9593661 rows, 6.14 GiB in 10.815726 sec., 887010.3588053174 rows/sec., 581.03 MiB/sec.
[cube] 2025.12.08 04:48:58.935197 [ 3725 ] {a3a04416-b2a4-4ccb-8297-62904397364b} MemoryTracker: Query peak memory usage: 2.56 GiB.
...
┌─count()─┐
1. │ 1138363 │ -- 1.14 million
└─────────┘
```
It takes 10s and it uses `low_cardinality_key_string_two_level`, great!
Now let's try the nested one:
```
:) select count() from (select stackMap.stack as stack, sum(stackMap.value) from nested_stacks array join stackMap group by stack)
...
[cube] 2025.12.08 04:49:46.395260 [ 802 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} Aggregator: Aggregation method: key_string_two_level
...
[cube] 2025.12.08 04:50:02.223290 [ 3725 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} executeQuery: Read 3646 rows, 7.13 GiB in 15.905909 sec., 229.22298876474147 rows/sec., 458.94 MiB/sec.
[cube] 2025.12.08 04:50:02.223459 [ 3725 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} MemoryTracker: Query peak memory usage: 2.73 GiB.
...
┌─count()─┐
1. │ 1138363 │ -- 1.14 million
└─────────┘
```
It gives the same answer, but it takes 16s compared to 10s before and it uses `key_string_two_level`.
### Which ClickHouse versions are affected?
Presumably all of them, but I'm on v25.8.12.129.
### How to reproduce
See above.
### Expected behavior
_No response_
### Error message and/or stacktrace
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.