apache / apache/pinot

[multistage][discuss] window agg with same partition but with/without order by

Open
#11,627 6 comments 0 reactions 0 assignees View on GitHub
feature multi-stage
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

queries with the same partition but a missing or non-missing order-by clause:
```
SELECT
ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) AS row_num,
COUNT(*) OVER (PARTITION BY a) AS cnt
from tbl
```
should be allowed, b/c both are partitioned by a
1. can we do it within the same window?

3. if not, can we create a plan similar to writing the SQL as the following automatically
```
WITH tmp AS (
SELECT
ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) AS row_num
FROM tbl
)

SELECT
row_num, COUNT(*) OVER (PARTITION BY a) AS cnt
FROM tmp
```

3. and when execute: can we optimize out the exchange
```
LogicalWindow(window#0=[window(partition {3} aggs [COUNT()])])
PinotLogicalExchange(distribution=[hash[3]]) <-- get rid of this exchange?
LogicalWindow(window#0=[window(partition {3} order by [10] rows between UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])])
PinotLogicalSortExchange(distribution=[hash[3]], collation=[[10]], isSortOnSender=[false], isSortOnReceiver=[true])
LogicalTableScan(table=[[tbl]])
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.