Support multiple count distinct
- Dominant language
- Rust
- Stars
- 1.8k
- Forks
- 241
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 22
Description
**Is your feature request related to a problem? Please describe.**
```
25/10/26 18:24:46 WARN AuronConverters: Falling back exec: HashAggregateExec: assertion failed: partial AggregateExec is not native
25/10/26 18:24:57 WARN AuronConverters: Falling back exec: HashAggregateExec: assertion failed
25/10/26 18:24:59 WARN AuronConverters: Falling back exec: HashAggregateExec: assertion failed: partial AggregateExec is not native
```
```sql
SELECT
region,
year,
COUNT(DISTINCT product) AS distinct_product_count,
COUNT(DISTINCT region) AS distinct_region_count
FROM (
SELECT
CASE WHEN id % 2 = 0 THEN 'Region1' ELSE 'Region2' END AS region,
CASE WHEN id % 3 = 0 THEN 'ProductA' ELSE 'ProductB' END AS product,
2023 AS year,
id * 100 AS sales_amount
FROM RANGE(10)
) AS sales
GROUP BY region, year;
```
Because Spark rule RewriteDistinctAggregates will rewrite multiple COUNT DISTINCT and add gid filter, Auron convertMoreAggregateExpr restricts the filter to be empty.
https://github.com/apache/auron/blob/3eb0e7353d0e280ddc7ae3dab77b8206c31dfb76/spark-extension-shims-spark/src/main/scala/org/apache/spark/sql/auron/ShimsImpl.scala#L558-L559
https://github.com/apache/spark/blob/65755bde3cb8b09bfaa8776fe6b4a182fd74ee0b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala#L249-L251
---
[SPARK-34882][SQL] Replace if with filter clause in RewriteDistinctAggregates
https://issues.apache.org/jira/browse/SPARK-34882
Fix Version/s: 3.2.0
**Describe the solution you'd like**
**Describe alternatives you've considered**
**Additional context**
Contributor guide
Assessment
This issue has not been assessed yet.