Provide a way to group by an entire multi value field
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Currently, Pinot explodes multi value columns included in grouping sets. This is desirable for many use cases but differs from how some other sql engines behave. Provide a way to group by an entire multi value field instead of exploding it, such as a transform function.
Here is an example to illustrate:
For a table with a multi value string array column and long column with the following values:
```
['hi', 'hello'], 3
['bye'], 4
['good', 'bye'], 2
```
And the query and results:
```
select string_array_col, count(*) from table group by string_array_col;
```
Results:
```
'hi', 1
'bye', 2
'good', 1
'hello', 1
```
Provide a transform function so that we can group by the entire array, for example:
```
select joinmv(string_array_col), count(*) from table group by joinmv(string_array_col)
['hi', 'hello'], 1
['bye'], 1
['good', 'bye'], 1
```
Contributor guide
Research direction
Start by tracing Pinot's handling of multi-value columns in grouping sets and the existing transform-function entry points. The work is done when a function such as joinmv lets grouping preserve each complete array, producing the three grouped results shown in the issue rather than exploding values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100