ClickHouse / ClickHouse/ClickHouse

huge performance impact when order by PK prefix

Open
#43,220 5 comments 0 reactions 0 assignees View on GitHub
comp-query-optimizer external performance
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

Version: 22.3.8.39-lts

How to reproduce:
```sql
CREATE TABLE login_log
(
`app_id` UInt8,
`company_id` UInt32,
`login_date` Date CODEC(DoubleDelta, LZ4),
`login_hour` UInt8,
`user_name` String CODEC(ZSTD(1)),
INDEX idx_sid company_id TYPE bloom_filter GRANULARITY 4
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(login_date)
ORDER BY (app_id, login_date)
SETTINGS index_granularity = 8192;

sql1:
SELECT company_id as userId,
login_date as loginDate,
login_hour as loginHour,
app_id as appId,
user_name
from user_login_log prewhere company_id= 1
where app_id= 5
and login_date< '2022-10-01'
ORDER by app_id DESC,
login_date DESC,
login_hour DESC
limit 1;
-- 1 rows in set. Elapsed: 5.847 sec. Processed 1.10 billion rows, 25.15 GB (188.53 million rows/s., 4.30 GB/s.)

sql2:
SELECT company_id as userId,
login_date as loginDate,
login_hour as loginHour,
app_id as appId,
user_name
from user_login_log prewhere company_id= 1
where app_id= 5
and login_date< '2022-10-01'
ORDER by login_date DESC,
login_hour DESC
limit 1;
-- 1 rows in set. Elapsed: 0.224 sec. Processed 3.75 million rows, 81.17 MB (16.76 million rows/s., 362.59 MB/s.)
```

the difference between sql1 and sql2 is only: sql1 have ORDER by app_id DESC, sql2 do not have. but sql1 scanned 25.15 GB data, sql2 only scanned 81.17 MB data.

what's the reason of the performance difference above?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.