apache / apache/pinot

Multiple Grouping Sets and Familia Support

Open
#8,060 3 comments 0 reactions 1 assignee Claimed by @atris View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

This issue tracks the development of the feature that brings multiple grouping sets to Pinot.

### What is A Grouping Set?

Consider the following query:

```
SELECT
brand,
segment,
SUM (quantity)
FROM
sales
GROUP BY
brand,
segment;
```

(brand, segment) represents a single grouping set.

A query using multiple grouping sets would be represented as:

```
SELECT
c1,
c2,
aggregate_function(c3)
FROM
table_name
GROUP BY
GROUPING SETS (
(c1, c2),
(c1),
(c2),
()
);
```

An equivalent query using UNION ALL would be:

```
SELECT
brand,
segment,
SUM (quantity)
FROM
sales
GROUP BY
brand,
segment

UNION ALL

SELECT
brand,
NULL,
SUM (quantity)
FROM
sales
GROUP BY
brand

UNION ALL

SELECT
NULL,
segment,
SUM (quantity)
FROM
sales
GROUP BY
segment

UNION ALL

SELECT
NULL,
NULL,
SUM (quantity)
FROM
sales;
```

GROUPING SETS also allows empty sets () which is equivalent of SELECT * FROM foo;

### CUBE and ROLLUP

`CUBE(c1, c2, c3) ` generates:

```
(c1, c2, c3)
(c1, c2)
(c2, c3)
(c1,c3)
(c1)
(c2)
(c3)
()
```

`ROLLUP(c1, c2,c3)` generates:

```
(c1, c2, c3)
(c1, c2)
(c1)
()
```

ROLLUP generates groups in hierarchy vs. CUBE generating all groups.

### Design

A design document shall soon be published but the design theme will be to use the swim lane concept introduced in the FILTER PR. An important design goal is to avoid rescans.

### Implementation Plan

The implementation plan will be to first support ROLLUP, then CUBE and then generic GROUPING sets.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.