ClickHouse / ClickHouse/ClickHouse

after change storage policy from cold_disk to s3, weird merge

Open
#42,519 1 comment 0 reactions 0 assignees View on GitHub
comp-ttl external question
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

#### version
22.9.2
#### queation
out old storage policy is hot_to_cold, it store data into disk.
new storage policy is hot_to_cold_s3 it store data into s3.
![截屏2022-10-20 上午11 24 27](https://user-images.githubusercontent.com/52686065/196849147-fb9de0d9-1ff0-4dc9-a210-ce22d38e64c4.png)

after change storage policy, there are a lot of weird system.merges:
it read data from cold disk, and then write back to it.
![截屏2022-10-20 上午9 47 20](https://user-images.githubusercontent.com/52686065/196849592-ef04f9db-3cff-4bc5-bf79-17ba642893eb.png)

after kill this mutation, the weird merge is closed.
![截屏2022-10-20 上午10 59 18](https://user-images.githubusercontent.com/52686065/196849818-f0e4814a-1d22-4665-9df9-539de2342b6f.png)

#### create table sql
old :
```
CREATE TABLE default.jaeger_spans_local
(
`timestamp` DateTime('UTC') CODEC(Delta(4), ZSTD(1)),
)
ENGINE = MergeTree
PARTITION BY toDate(timestamp)
ORDER BY (-toUnixTimestamp(timestamp))
TTL timestamp + toIntervalDay(7), timestamp + toIntervalDay(1) TO DISK 'cold_disk'
SETTINGS index_granularity = 1024, storage_policy = 'hot_to_cold'
```

new:
```
CREATE TABLE default.jaeger_spans_local
(
`timestamp` DateTime('UTC') CODEC(Delta(4), ZSTD(1)),
)
ENGINE = MergeTree
PARTITION BY toDate(timestamp)
ORDER BY (-toUnixTimestamp(timestamp))
TTL timestamp + toIntervalDay(7), timestamp + toIntervalDay(1) TO DISK 'cold_disk_s3'
SETTINGS index_granularity = 1024, storage_policy = 'hot_to_cold_s3'
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.