apache / apache/datafusion

`avg` over `Duration` panics with "attempt to add with overflow" in debug builds

Open Beginner friendly
#24,894 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

`DurationAvgAccumulator` (`datafusion/functions-aggregate/src/average.rs`) keeps its running sum with plain `+=` in `update_batch` / `merge_batch` and plain `-` in `retract_batch`. When the sum of the input durations does not fit in `i64` this panics in debug builds and wraps silently in release builds.

The `sum` aggregate and the other `avg` accumulators deliberately wrap on overflow (see the comment on `add_avg_sum`: "Wraps on overflow, matching the `sum` aggregate and `arrow::compute::sum`"), so the Duration accumulator is the odd one out.

### To Reproduce

Debug build of `datafusion-cli`:

```sql
SELECT avg(x) OVER (ROWS BETWEEN 1 PRECEDING AND CURRENT ROW)
FROM (VALUES (arrow_cast(9223372036854775807, 'Duration(Second)')),
(arrow_cast(9223372036854775807, 'Duration(Second)'))) t(x);
```

```text
thread 'main' panicked at datafusion/functions-aggregate/src/average.rs:887:13:
attempt to add with overflow
```

The plain aggregate `SELECT avg(x) FROM ...` with the same values hits the same line.

### Expected behavior

Consistent with `sum` and the other `avg` accumulators: wrap, never panic.

### Additional context

Found while running a corpus of extreme-value literals against a debug build of `datafusion-cli`.

Contributor guide

Open the contributing guide

Research direction

Start in datafusion/functions-aggregate/src/average.rs and inspect DurationAvgAccumulator::update_batch, merge_batch, and retract_batch alongside add_avg_sum. Run the provided debug-build datafusion-cli SQL reproduction, then verify that duration averages wrap on overflow like sum and the other avg accumulators without panicking.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.