ClickHouse / ClickHouse/ClickHouse
Alter table attach / move / replace partition should clean TTL info from the attached parts
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
https://fiddle.clickhouse.com/3dd9704d-2a0f-475a-96ff-49c706cd7390
data is removed from the table with `TTL ts + interval 100 day` because the attached part has another TTL info `TTL ts + interval 1 hour`
```sql
CREATE TABLE users (uid Int64, ts DateTime)
ENGINE=MergeTree partition by toYYYYMMDD(ts) order by uid
TTL ts + interval 1 hour;
CREATE TABLE users_history (uid Int64, ts DateTime)
ENGINE=MergeTree partition by toYYYYMMDD(ts) order by uid
TTL ts + interval 100 day;
insert into users select number, now()- interval 59 minute - interval 58 second from numbers(10000);
alter table users move partition tuple(toYYYYMMDD(now())) to table users_history;
select count() from users_history;
┌─count()─┐
│ 10000 │
└─────────┘
select sleep(3) format Null;
optimize table users_history final;
select count() from users_history;
┌─count()─┐
│ 0 │ -- data is removed by TTL
└─────────┘
select delete_ttl_info_min, delete_ttl_info_max, rows, name
from system.parts
where table = 'users_history' and active ;
┌─delete_ttl_info_min─┬─delete_ttl_info_max─┬─rows─┬─name───────────┐
│ 1970-01-01 00:00:00 │ 1970-01-01 00:00:00 │ 0 │ 20230605_1_1_1 │
└─────────────────────┴─────────────────────┴──────┴────────────────┘
```
Or TTL remover should check the current TTL rule on the part somehow? Validating a single row?
The same behavior with a single table (materialize_ttl_after_modify=0) https://fiddle.clickhouse.com/00f5d9e3-3548-4e3e-a40c-096495c67b15
```sql
CREATE TABLE users (uid Int64, ts DateTime)
ENGINE=MergeTree partition by toYYYYMMDD(ts) order by uid
TTL ts + interval 1 hour;
insert into users select number, now()- interval 59 minute - interval 58 second from numbers(10000);
select count() from users;
┌─count()─┐
│ 10000 │
└─────────┘
set materialize_ttl_after_modify =0;
alter table users modify TTL ts + INTERVAL 100 day;
select sleep(3) format Null;
optimize table users final;
select count() from users;
┌─count()─┐
│ 0 │
└─────────┘
```
Contributor guide
Assessment
This issue has not been assessed yet.