apache / apache/doris

[Enhancement] Support multiple DISTINCT aggregates for collect_list and array_agg

Open
#65,244 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Description

When a single GROUP BY contains more than one DISTINCT aggregate and any of them
is `collect_list(distinct ...)` or `array_agg(distinct ...)`, planning fails with:

errCode = 2, ... can't support multi distinct.

`CheckMultiDistinct` rejects any distinct aggregate that does not implement
`SupportMultiDistinct`, and `collect_list` / `array_agg` do not implement it.
So queries like these cannot run:

select g, collect_list(distinct a), collect_list(distinct b) from t group by g;
select g, array_agg(distinct a), array_agg(distinct b) from t group by g;

Other aggregates (count, sum, sum0, group_concat) already support this through
the multi-distinct framework. `collect_list` and `array_agg` should too, via new
`multi_distinct_collect_list` and `multi_distinct_array_agg` functions.

### Solution

Add `multi_distinct_collect_list` / `multi_distinct_array_agg` and make
`CollectList` / `ArrayAgg` implement `SupportMultiDistinct`, plugging into the
existing multi-distinct rewrite (BE + FE).

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by tracing CheckMultiDistinct and the existing multi-distinct implementations for count, sum, sum0, and group_concat. Then inspect CollectList and ArrayAgg across the FE and BE paths. Done means the new multi_distinct_collect_list and multi_distinct_array_agg functions participate in the rewrite and both DISTINCT aggregate query examples plan and run successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.