apache / apache/datafusion

Support for aggregate functions on intervals

Open
#23,085 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

In postgres you can have the following query:
```sql
with test_values as (
select * from (values (interval '1 second'), (interval '1 year'), (interval '1 month')) as t(value)
)
select avg(value) from test_values;

>> 0 years 4 mons 10 days 0 hours 0 mins 0.333333 secs

with test_values as (
select * from (values (interval '1 second'), (interval '1 year'), (interval '1 month')) as t(value)
)
select sum(value) from test_values;

>> 1 years 1 mons 0 days 0 hours 0 mins 1.0 secs
```

It would be nice if Datafusion also had support for it. Datafusion current output:
```sql
> with test_values as (
select * from (values (interval '1 second'), (interval '1 year'), (interval '1 month')) as t(value)
)
select avg(value) from test_values;
Error during planning: Internal error: Function 'avg' failed to match any signature, errors: Error during planning: Function 'avg' requires Decimal, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'avg' requires Duration, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'avg' requires Float64, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano))..
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues. No function matches the given name and argument types 'avg(Interval(MonthDayNano))'. You might need to add explicit type casts.
Candidate functions:
avg(Decimal)
avg(Duration)
avg(Float64)

> with test_values as (
select * from (values (interval '1 second'), (interval '1 year'), (interval '1 month')) as t(value)
)
select sum(value) from test_values;
Error during planning: Internal error: Function 'sum' failed to match any signature, errors: Error during planning: Function 'sum' requires Decimal, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'sum' requires UInt64, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'sum' requires Int64, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'sum' requires Float64, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano)).,Error during planning: Function 'sum' requires Duration, but received Interval(MonthDayNano) (DataType: Interval(MonthDayNano))..
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues. No function matches the given name and argument types 'sum(Interval(MonthDayNano))'. You might need to add explicit type casts.
Candidate functions:
sum(Decimal)
sum(UInt64)
sum(Int64)
sum(Float64)
sum(Duration)
```

### Describe the solution you'd like

_No response_

### Describe alternatives you've considered

aggregating epoch
```sql
with test_values as (
select * from (values (interval '1 second'), (interval '1 year'), (interval '1 month')) as t(value)
)
select sum(extract(epoch from value)) from test_values;
````

it works but it wont be a duration/interval anymore

### Additional context

Very nice for time series operations. for example if you would like to know the total duration of present gaps of a time series

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by reproducing the PostgreSQL comparison queries and tracing how DataFusion resolves avg and sum signatures for Interval(MonthDayNano); done means both aggregates accept intervals and return interval/duration results consistent with the requested behavior, with coverage for the supplied values.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.