ClickHouse / ClickHouse/ClickHouse
old partitions/folders not getting deleted
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
### Company or project name
_No response_
### Question
Hi , I fetch historical financial data and update daily.. Problem is i want old data to be deleted and only latest data to be available , but ReplacingMergeTree is replacing old data.. but the folders inside clickhouse/store is increasing day by day daily.. even if i drop database the store folder has lots of data. How to properly replace old data and remove old paritions/folders created by clickhouse.
```
CREATE TABLE ohlc
(
symbol LowCardinality(String),
timeframe UInt8,
timestamp DateTime('UTC'),
open Float32,
high Float32,
low Float32,
close Float32,
volume Float64,
symbol_tf String MATERIALIZED concat(symbol, '_', toString(timeframe)),
month UInt32 MATERIALIZED toYYYYMM(timestamp)
)
ENGINE = ReplacingMergeTree(timestamp)
PARTITION BY symbol_tf
PRIMARY KEY (symbol_tf, timestamp)
ORDER BY (symbol_tf, timestamp)
TTL timestamp + INTERVAL 2 MONTH DELETE WHERE timeframe = 1,
timestamp + INTERVAL 3 MONTH DELETE WHERE timeframe = 3,
timestamp + INTERVAL 4 MONTH DELETE WHERE timeframe = 5,
timestamp + INTERVAL 6 MONTH DELETE WHERE timeframe = 10,
timestamp + INTERVAL 12 MONTH DELETE WHERE timeframe = 15,
timestamp + INTERVAL 24 MONTH DELETE WHERE timeframe = 30,
timestamp + INTERVAL 36 MONTH DELETE WHERE timeframe = 60,
timestamp + INTERVAL 120 MONTH DELETE WHERE timeframe = 99
SETTINGS
index_granularity = 8192,
use_minimalistic_part_header_in_zookeeper = 1,
min_bytes_for_wide_part = 0;
```
Contributor guide
Assessment
This issue has not been assessed yet.