ClickHouse / ClickHouse/ClickHouse
Regression: no partition pruning when ARRAY JOIN is used together with certain predicates
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Company or project name**
My company is developing various analytical tools based on ClickHouse. After recent upgrade to 24.3.6.47 we've faced rapid performance degradation of some queries. Further investigation revealed that the issue is still reproducible in the most recent release.
**Describe the unexpected behaviour**
Example: https://fiddle.clickhouse.com/a1e35a92-8a8c-42c0-a6e3-b8166b7ecf47
```sql
create table test (
d DateTime,
arr Array(LowCardinality(String))
)
ENGINE = MergeTree
PARTITION BY toStartOfDay(d)
ORDER BY tuple();
insert into test (d, arr) VALUES ('2024-10-01 01:00:00', ['1']),('2024-10-02 01:00:00', ['2']);
explain indexes=1
select count()
from test array join arr
where true and (arr = '1') AND (d >= '2024-10-01 00:59:00') AND (d < '2024-10-01 01:01:00');
-- ClickHouse ignores partition key and is eager to perform full scan
```
This looks like incomplete implementation as optimizer can be enabled again by **either** of the following changes:
1. Drop leading `true` in the `WHERE` clause.
2. Drop `arr = '1'` clause from `WHERE`.
3. Change type from `arr Array(LowCardinality(String))` to `arr Array(String)`
**How to reproduce**
* Reproducible in the latest release.
* Either interface.
* allow_experimental_analyzer setting doesn't alter the behavior
**Expected behavior**
It's expected that partitioning pruning will take place for the second query as well.
**Error message and/or stacktrace**
None
**Additional context**
Version `23.8.9.54` is free from this issue, didn't bisect though.
Contributor guide
Assessment
This issue has not been assessed yet.