apache / apache/datafusion

SQL Logical Plan Drops ordering within UDAFs

Open
#7,531 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

SQL supports ordering within an aggregate function, e.g. `SUM(a order by b desc)`. DataFusion's logical planner does correctly identify this for built-in aggregates, e.g. `sum()`, `max()`, `min()`. However, for UDAFs, it seems to be dropped.

I've written up an example here: https://github.com/ArroyoSystems/arrow-datafusion/blob/92e92b9e971bc407b39de2ce8c9bc793355168f7/datafusion-examples/examples/simple_udaf.rs#L174.

The output is
```
built-in aggregate Logical plan, has ORDER BY:
Projection: SUM(t.a) ORDER BY [t.a DESC NULLS FIRST]
Aggregate: groupBy=[[]], aggr=[[SUM(t.a) ORDER BY [t.a DESC NULLS FIRST]]]
TableScan: t
UDAF Logical plan, missing ORDER BY:
Projection: geo_mean(t.a)
Aggregate: groupBy=[[]], aggr=[[geo_mean(t.a)]]
TableScan: t
```

### To Reproduce

Register a UDAF, write a query with ordering inside it, and inspect the logical plan.

### Expected behavior

The UDAF expression should have a non-empty `order_by` clause.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with datafusion-examples/examples/simple_udaf.rs around line 174 and run the UDAF example described in the issue. Compare its logical plan with the built-in aggregate plan; done means the UDAF expression retains a non-empty order_by clause.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.