ClickHouse / ClickHouse/ClickHouse
Slow materialized view degrades latency of inserts not triggering it (24.1)
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Describe the unexpected behaviour**
If there are two materialized views on the source table triggering on different `WHERE` conditions then having a heavy query somewhere down in the second pipeline (in a cascaded materialized view) will slow down processing of inserts on the first pipeline. The regression introduced in 23.12 (actually it seems initially it was introduced in 23.10 then fixed in 23.11 then reintroduced in 23.12). Beforehand the pipelines used to work in isolation not affecting one another.
**How to reproduce**
https://fiddle.clickhouse.com/0a4f43f2-b742-4534-b215-8affa89d0221
```
create table source(type String) engine=MergeTree order by type;
create view v_heavy as
with nums as (select number from numbers(1e5))
select count(*) n from (select number from numbers(1e5) n1 cross join nums);
create table target1(type String) engine=MergeTree order by type;
create table target2(type String) engine=MergeTree order by type;
create table post_target1(type String) engine=MergeTree order by type;
create table post_target2(type String) engine=MergeTree order by type;
create materialized view vm_target1 to target1 as select * from source where type='one';
create materialized view vm_target2 to target2 as select * from source where type='two';
create materialized view vm_post_target1 to post_target1 as select * from target1 where type='one';
create materialized view vm_post_target2 to post_target2 as select * from target2 where type='two' and (select n from v_heavy)>0;
insert into source values('one'); -- slow
```


**Expected behavior**
Only queries from the materialized views directly involved in the processing of current "insert" should be executed (affecting latency and resources consumption).
**Additional context**
I'm aware of a possible solution to split the source table into two. But it comes with additional hustle and besides used to exhibit correct behaviour before version 23.12
Contributor guide
Assessment
This issue has not been assessed yet.