apache / apache/datafusion

Extract common aggregation spill logic

Open
#24,317 0 comments 0 reactions 1 assignee Claimed by @2010YOUY01 View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

The spilling path added for `OrderedSingleAggregateStream` in #24259 is largely similar to the spilling paths used by other aggregation modes. Keeping separate copies of the same spill mechanics makes future fixes and changes easier to miss and can cause the implementations to drift.

This is follow-up work from https://github.com/apache/datafusion/pull/24259#discussion_r3757233396 and part of #22710.

### Describe the solution you'd like

Identify the semantically identical parts of the aggregation spilling paths and extract them into shared logic that can be reused by the relevant aggregation streams.

Keep mode-specific policies and state transitions explicit in each dedicated stream; only the common spill mechanics should be shared.

### Describe alternatives you've considered

### Additional context

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.