Vectorize window functions (`GroupsAccumulator`-style)
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
[More info to be added later]
DataFusion has quite good support for window functions, but (part of) the window function implementation is not yet (I guess mostly because we don't run popular benchmarks with window functions, tpch and clickbench don't have any).
We can try giving it a similar treatment as aggregates (e.g. GroupsAccumulator).
### Describe the solution you'd like
Design a strategy and implement vectorized code for window functions (sum/count, etc) where possible.
Benchmark the improvement.
### Describe alternatives you've considered
_No response_
### Additional context
[Add a profiling image / info here]
Contributor guide
Research direction
Start by reviewing DataFusion’s existing window-function implementation and the GroupsAccumulator-style aggregate approach. Identify which operations, such as sum and count, can be vectorized, then establish benchmark coverage and measure the improvement against the current implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100