Improve aggregations through an acyclic visitor pattern
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
This issue is a longer description for a project originally proposed here: https://github.com/rapidsai/cudf/issues/10432#issuecomment-1067560695
## High-level overview
Several features in libcudf are implemented using “aggregations”: operations that act on multiple pieces of data. Aggregations enable a wide range of computations, including scans over a column of data, reductions over a column to find quantiles or standard deviations, reductions over grouped data, rolling (windowed) averages, or segmented reductions that can find the minimum value of each list in a column. Real-world examples include computing a rolling average of sales through the year, or average sales grouped by the day of the week. Some aggregations are simple to implement (like a sum), while others require data pre-processing (like sum-of-squares, which squares each element before performing the sum across elements) or data post-processing (like a mean, which sums across elements and then divides by the number of elements). Currently, the code in libcudf implements a [visitor pattern](https://en.wikipedia.org/wiki/Visitor_pattern) for the groupby and rolling code paths using aggregation, but not for other paths such as reductions and segmented reductions.
## Proposed changes
This issue proposes to:
- implement the visitor pattern style of groupby/rolling for additional forms of aggregation such as reductions and segmented reductions
- improve consistency and code reuse for aggregations across the library
The unified design should be benchmarked. Existing tests should be adapted to ensure complete coverage. Aggregation design documentation should be written for the libcudf developer docs so these patterns can be recognized and reused where appropriate.
Ideally, we would like to use an _acyclic visitor pattern_ to reduce the amount of boilerplate code needed. References:
- https://condor.depaul.edu/dmumaugh/OOT/Design-Principles/acv.pdf
- https://codecrafter.blogspot.com/2012/12/the-acyclic-visitor-pattern.html
See also this table overview of [aggregation algorithms and operations](https://gist.github.com/codereport/b747c8cad4026fa1defc422411afbcbc).
Contributor guide
Assessment
This issue has not been assessed yet.