ClickHouse / ClickHouse/ClickHouse
huge performance impact when order by PK prefix
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
Version: 22.3.8.39-lts
How to reproduce:
```sql
CREATE TABLE login_log
(
`app_id` UInt8,
`company_id` UInt32,
`login_date` Date CODEC(DoubleDelta, LZ4),
`login_hour` UInt8,
`user_name` String CODEC(ZSTD(1)),
INDEX idx_sid company_id TYPE bloom_filter GRANULARITY 4
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(login_date)
ORDER BY (app_id, login_date)
SETTINGS index_granularity = 8192;
sql1:
SELECT company_id as userId,
login_date as loginDate,
login_hour as loginHour,
app_id as appId,
user_name
from user_login_log prewhere company_id= 1
where app_id= 5
and login_date< '2022-10-01'
ORDER by app_id DESC,
login_date DESC,
login_hour DESC
limit 1;
-- 1 rows in set. Elapsed: 5.847 sec. Processed 1.10 billion rows, 25.15 GB (188.53 million rows/s., 4.30 GB/s.)
sql2:
SELECT company_id as userId,
login_date as loginDate,
login_hour as loginHour,
app_id as appId,
user_name
from user_login_log prewhere company_id= 1
where app_id= 5
and login_date< '2022-10-01'
ORDER by login_date DESC,
login_hour DESC
limit 1;
-- 1 rows in set. Elapsed: 0.224 sec. Processed 3.75 million rows, 81.17 MB (16.76 million rows/s., 362.59 MB/s.)
```
the difference between sql1 and sql2 is only: sql1 have ORDER by app_id DESC, sql2 do not have. but sql1 scanned 25.15 GB data, sql2 only scanned 81.17 MB data.
what's the reason of the performance difference above?
Contributor guide
Assessment
This issue has not been assessed yet.