ClickHouse / ClickHouse/ClickHouse

Nested LowCardinality does not engage low_cardinality_key_string for aggregations

Open
#91,647 1 comment 0 reactions 0 assignees View on GitHub
comp-aggregation external unexpected behaviour
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Company or project name

_No response_

### Describe the unexpected behaviour

I have a table that as a bunch of columns and then a mapping from stacks to their counts to make flamegraphs. Stripping everything else, it can be expressed in two ways:

1. As a flat table (not ideal if you have many other columns that would need duplication):

```
:) create table flat_stacks (stack LowCardinality(String), value UInt64 CODEC(ZSTD(1))) engine = MergeTree
```

2. As a nested column:

```
:) create table nested_stacks (stackMap Nested(stack LowCardinality(String), value UInt64) CODEC(ZSTD(1))) engine = MergeTree
```

Let's insert the same data into both:

```
:) insert into flat_stacks select stackMap.stack as stack, stackMap.value as value from welp array join stackMap
:) insert into nested_stacks select stackMap.stack, stackMap.value from welp
```

Here's how it looks like in terms of data on disk:

```
┌─table─────────┬─column─────────┬────rows─┬─uncompressed─┬─compressed─┬─uncompressed_per_row─┬─compressed_per_row─┬─compression_ratio─┐
1. │ flat_stacks │ stack │ 9593661 │ 7661036261 │ 381306586 │ 798.55 │ 39.74 │ 20.09 │
2. │ flat_stacks │ value │ 9593661 │ 76749288 │ 8161820 │ 8.00 │ 0.85 │ 9.40 │
3. │ nested_stacks │ stackMap.stack │ 3646 │ 7345456902 │ 236466574 │ 2014661.79 │ 64856.43 │ 31.06 │
4. │ nested_stacks │ stackMap.value │ 3646 │ 76749288 │ 8042077 │ 21050.27 │ 2205.72 │ 9.54 │
└───────────────┴────────────────┴─────────┴──────────────┴────────────┴──────────────────────┴────────────────────┴───────────────────┘
```

Now let's query the flat table:

```
:) select count() from (select stack, sum(value) from flat_stacks group by stack)
...
[cube] 2025.12.08 04:48:48.202084 [ 803 ] {a3a04416-b2a4-4ccb-8297-62904397364b} Aggregator: Aggregation method: low_cardinality_key_string_two_level
...
[cube] 2025.12.08 04:48:58.935135 [ 3725 ] {a3a04416-b2a4-4ccb-8297-62904397364b} executeQuery: Read 9593661 rows, 6.14 GiB in 10.815726 sec., 887010.3588053174 rows/sec., 581.03 MiB/sec.
[cube] 2025.12.08 04:48:58.935197 [ 3725 ] {a3a04416-b2a4-4ccb-8297-62904397364b} MemoryTracker: Query peak memory usage: 2.56 GiB.
...

┌─count()─┐
1. │ 1138363 │ -- 1.14 million
└─────────┘
```

It takes 10s and it uses `low_cardinality_key_string_two_level`, great!

Now let's try the nested one:

```
:) select count() from (select stackMap.stack as stack, sum(stackMap.value) from nested_stacks array join stackMap group by stack)
...
[cube] 2025.12.08 04:49:46.395260 [ 802 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} Aggregator: Aggregation method: key_string_two_level
...
[cube] 2025.12.08 04:50:02.223290 [ 3725 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} executeQuery: Read 3646 rows, 7.13 GiB in 15.905909 sec., 229.22298876474147 rows/sec., 458.94 MiB/sec.
[cube] 2025.12.08 04:50:02.223459 [ 3725 ] {11c96ae9-2fd5-4d85-8684-3d5d063bba22} MemoryTracker: Query peak memory usage: 2.73 GiB.
...

┌─count()─┐
1. │ 1138363 │ -- 1.14 million
└─────────┘
```

It gives the same answer, but it takes 16s compared to 10s before and it uses `key_string_two_level`.

### Which ClickHouse versions are affected?

Presumably all of them, but I'm on v25.8.12.129.

### How to reproduce

See above.

### Expected behavior

_No response_

### Error message and/or stacktrace

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.