apache / apache/datafusion

`single_distinct_to_groupby` not applied to DataFrame API count(DISTINCT)

Open
#23,401 3 comments 0 reactions 1 assignee Claimed by @gmhelmold View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

The `SingleDistinctToGroupBy` optimizer rule rewrites a grouped count(DISTINCT col) into a cheaper nested double aggregate. It fires for plans built by the SQL planner but not for the logically-equivalent plan built by the DataFrame API, so the DataFrame path runs materially slower for the same query.

The divergence is caused by how each front-end places the output alias:
- SQL (count(DISTINCT v) AS n): the alias becomes an outer Projection; Aggregate.aggr_expr holds a bare Expr::AggregateFunction.
- DataFrame API (count(col("v")).distinct().alias("n")): the alias wraps the aggregate directly, so aggr_expr holds Expr::Alias(AggregateFunction).

`is_single_distinct_agg` only matches a bare Expr::AggregateFunction and returns false for the aliased shape, so the rule never fires for the DataFrame API.

### To Reproduce

_No response_

### Expected behavior

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.