[Enhancement] Support multiple DISTINCT aggregates for collect_list and array_agg
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
When a single GROUP BY contains more than one DISTINCT aggregate and any of them
is `collect_list(distinct ...)` or `array_agg(distinct ...)`, planning fails with:
errCode = 2, ... can't support multi distinct.
`CheckMultiDistinct` rejects any distinct aggregate that does not implement
`SupportMultiDistinct`, and `collect_list` / `array_agg` do not implement it.
So queries like these cannot run:
select g, collect_list(distinct a), collect_list(distinct b) from t group by g;
select g, array_agg(distinct a), array_agg(distinct b) from t group by g;
Other aggregates (count, sum, sum0, group_concat) already support this through
the multi-distinct framework. `collect_list` and `array_agg` should too, via new
`multi_distinct_collect_list` and `multi_distinct_array_agg` functions.
### Solution
Add `multi_distinct_collect_list` / `multi_distinct_array_agg` and make
`CollectList` / `ArrayAgg` implement `SupportMultiDistinct`, plugging into the
existing multi-distinct rewrite (BE + FE).
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Contributor guide
Research direction
Start by tracing CheckMultiDistinct and the existing multi-distinct implementations for count, sum, sum0, and group_concat. Then inspect CollectList and ArrayAgg across the FE and BE paths. Done means the new multi_distinct_collect_list and multi_distinct_array_agg functions participate in the rewrite and both DISTINCT aggregate query examples plan and run successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100