ClickHouse / ClickHouse/ClickHouse

Regression: no partition pruning when ARRAY JOIN is used together with certain predicates

Open
#71,218 2 comments 0 reactions 0 assignees View on GitHub
comp-query-optimizer external unfinished code
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

**Company or project name**

My company is developing various analytical tools based on ClickHouse. After recent upgrade to 24.3.6.47 we've faced rapid performance degradation of some queries. Further investigation revealed that the issue is still reproducible in the most recent release.

**Describe the unexpected behaviour**
Example: https://fiddle.clickhouse.com/a1e35a92-8a8c-42c0-a6e3-b8166b7ecf47

```sql
create table test (
d DateTime,
arr Array(LowCardinality(String))
)
ENGINE = MergeTree
PARTITION BY toStartOfDay(d)
ORDER BY tuple();

insert into test (d, arr) VALUES ('2024-10-01 01:00:00', ['1']),('2024-10-02 01:00:00', ['2']);

explain indexes=1
select count()
from test array join arr
where true and (arr = '1') AND (d >= '2024-10-01 00:59:00') AND (d < '2024-10-01 01:01:00');

-- ClickHouse ignores partition key and is eager to perform full scan
```

This looks like incomplete implementation as optimizer can be enabled again by **either** of the following changes:
1. Drop leading `true` in the `WHERE` clause.
2. Drop `arr = '1'` clause from `WHERE`.
3. Change type from `arr Array(LowCardinality(String))` to `arr Array(String)`

**How to reproduce**
* Reproducible in the latest release.
* Either interface.
* allow_experimental_analyzer setting doesn't alter the behavior

**Expected behavior**
It's expected that partitioning pruning will take place for the second query as well.

**Error message and/or stacktrace**
None

**Additional context**
Version `23.8.9.54` is free from this issue, didn't bisect though.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.