apache / apache/datafusion

Epic: Ordered Set Aggregate Functions

Open
#12,824 9 comments 4 reactions 0 assignees View on GitHub
enhancement PROPOSAL EPIC
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

DataFusion doesn't support ordered-set aggregate functions yet.

Those functions are supported in [Postgres](https://www.postgresql.org/docs/9.4/functions-aggregate.html), [DuckDB](https://duckdb.org/docs/sql/functions/aggregates#ordered-set-aggregate-functions), so I think we should support them in this repo as the core functions

![Screenshot 2024-10-09 at 8 49 19 AM](https://github.com/user-attachments/assets/d249a617-1efd-4259-93cc-f23781b29a1b)

> All the aggregates listed in [Table 9-51](https://www.postgresql.org/docs/9.4/functions-aggregate.html#FUNCTIONS-ORDEREDSET-TABLE) ignore null values in their sorted input. For those that take a fraction parameter, the fraction value must be between 0 and 1; an error is thrown if not. However, a null fraction value simply produces a null result.

Related PR, https://github.com/apache/datafusion/issues/11732.

We have `approx_percentile_cont` and `approx_percentile_cont_with_weight` already, maybe we can start from these two functions

- [x] approx_percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression)
- [x] approx_percentile_cont_with_weight(fraction) WITHIN GROUP (ORDER BY sort_expression)

**Tasks to do**

- [x] Support WITHIN GROUP syntax
- #13511
- [ ] mode() WITHIN GROUP (ORDER BY sort_expression)
- [x] percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression)
- #17988
- [ ] percentile_cont(fractions) WITHIN GROUP (ORDER BY sort_expression)
- #18600
- [ ] percentile_disc(fraction) WITHIN GROUP (ORDER BY sort_expression)
- [ ] percentile_disc(fractions) WITHIN GROUP (ORDER BY sort_expression)
- [ ] Clarify our support of ordered-set aggregate functions
- #17805
- [x] #18109
- [x] https://github.com/apache/datafusion/issues/18279
- [x] https://github.com/apache/datafusion/issues/18280
- [x] https://github.com/apache/datafusion/issues/18281
- [ ] #18601

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing approx_percentile_cont and approx_percentile_cont_with_weight functions, along with the WITHIN GROUP syntax work in #13511. Choose one unchecked function from the task list and compare its required behavior with the PostgreSQL and DuckDB references. Done means the selected ordered-set aggregate is supported and its checklist item is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.