ClickHouse / ClickHouse/ClickHouse
Query targeting the primary keys is so slow
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
We have the following local table with more than 100 columns:
```SQL
CREATE TABLE IF NOT EXISTS my_db.my_table_local
(
Timestamp DateTime,
Date Date DEFAULT toDate(Timestamp),
ProjectId UInt64,
UserId UInt32,
SessionId UInt32,
PageNum UInt8,
...
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/my_db/my_table_local/{layer}-{shard}', '{replica}')
PARTITION BY toYYYYMM(Timestamp)
ORDER BY (ProjectId, Date, intHash32(UserId), SessionId, PageNum)
SAMPLE BY intHash32(UserId)
TTL Timestamp + INTERVAL 13 MONTH DELETE
SETTINGS storage_policy = 'move_from_ssd_to_hdd';
```
Running the following simple query on the distributed table ends up being so slow:
```SQL
SELECT *
FROM my_db.my_table_all
WHERE ProjectId =
AND UserId = AND SessionId = AND PageNum =
AND Date =
-- 1 rows in set. Elapsed: 197.097 sec. Processed 9.73 million rows, 117.48 MB (49.36 thousand rows/s., 596.05 KB/s.)
```
Other variations of this query are also so slow. For example:
```SQL
SELECT *
FROM my_db.my_table_all
WHERE ProjectId =
LIMIT 1
-- 1 rows in set. Elapsed: 3.860 sec. Processed 36.05 thousand rows, 87.80 MB (9.34 thousand rows/s., 22.75 MB/s.)
```
The performance of these queries is unexpected for me as I am adding exact filter conditions on the primary keys. Also by looking the number of processed rows, it feels that there is something wrong. We have way more complex queries that finish in about 200 ms on average!
We are using `clickhouse-server` version `21.7.11.3`.
Any idea why the queries are so slow and how to enhance them?
Contributor guide
Assessment
This issue has not been assessed yet.