Support multiple order aware aggregate functions in a query
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Today DataFusion supports three aggregate functions that can be "order aware": `ARRAY_AGG`, `FIRST_VALUE` and `LAST_VALUE`. This means that you can supply a `ORDER BY` clause to their argument, for example `FIRST_VALUE(x ORDER BY time)`.
Today, there be only one single order specified across ALL order aware aggregate functions
For example
```SQL
❯ create table t(x int, y int) as values (1, 1), (1, 2), (1, 1), (1, 4), (2, 20), (2, 10);;
0 rows in set. Query took 0.003 seconds.
❯ select x, first_value(x ORDER BY y) from t GROUP BY x;
+---+------------------+
| x | FIRST_VALUE(t.x) |
+---+------------------+
| 2 | 2 |
| 1 | 1 |
+---+------------------+
2 rows in set. Query took 0.004 seconds.
❯ select x, first_value(x ORDER BY y), first_value(x ORDER BY y DESC) from t GROUP BY x;
+---+------------------+-----------------+
| x | FIRST_VALUE(t.x) | LAST_VALUE(t.x) |
+---+------------------+-----------------+
| 1 | 1 | 1 |
| 2 | 2 | 2 |
+---+------------------+-----------------+
2 rows in set. Query took 0.004 seconds.
❯ select x, first_value(x ORDER BY y), first_value(x ORDER BY y DESC NULLS LAST) from t GROUP BY x;
This feature is not implemented: Conflicting ordering requirements in aggregate functions is not supported
```
### Describe the solution you'd like
There are a few designs proposed here: https://github.com/apache/arrow-datafusion/pull/8558#issuecomment-1862649886
We are working on a more detailed proposal
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by reading the designs proposed in PR 8558's issue comment and the more detailed proposal mentioned in this issue. Use the SQL examples as the acceptance cases: queries with different ORDER BY clauses for multiple order-aware aggregate functions should work without the conflicting-ordering error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100