ClickHouse / ClickHouse/ClickHouse

Projections are not very smart about datetime dimension.

Open
#26,369 3 comments 0 reactions 0 assignees View on GitHub
comp-projections external feature
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

```sql
create table ptest ( account UInt64, clip UInt64, ts DateTime, v Int64, play Int64, uid Int64 ) Engine = MergeTree order by tuple();

insert into ptest select number%3, number%7, toDateTime('2020-01-01 00:00:00') + number*100, number, number, number from numbers(10000000);

alter table ptest add projection pr1
(select account, clip, toDate(ts), sum(v) as loads, sum(play) as plays,
uniq(uid) as unique_loads group by account,clip,
toDate(ts));

alter table ptest materialize projection pr1;

set allow_experimental_projection_optimization = 1;
```

### Projection is used

```sql
select clip, sum(v), sum(play), uniq(uid)
from ptest 
where account=1 and toDate(ts) >= '2020-03-01' AND toDate(ts) <= '2020-04-01'
group by clip
format Null;

Processed 127.56 thousand rows

select clip, toDate(ts), sum(v), sum(play), uniq(uid)
from ptest 
where account=1 and toDate(ts) >= '2020-03-01' AND toDate(ts) <= '2020-04-01'
group by clip, toDate(ts)
format Null;

Processed 127.56 thousand rows
```

### Projection is not used
```sql
select clip, toYYYYMM(ts), sum(v), sum(play), uniq(uid)
from ptest 
where account=1 and toDate(ts) >= '2020-03-01' AND toDate(ts) <= '2020-04-01'
group by clip, toYYYYMM(ts)
format Null;

Processed 10.00 million rows

select clip, sum(v), sum(play), uniq(uid)
from ptest 
where account=1 and
toStartOfMonth(ts) >= toStartOfMonth(toDate('2020-03-01')) AND toStartOfMonth(ts) <= toStartOfMonth(toDate('2020-04-01'))
group by clip
format Null;

Processed 10.00 million rows
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.