citusdata / citusdata/citus

Reduce code duplication in aggregate logic

Open
#2,192 1 comment 0 reactions 0 assignees View on GitHub
technical debt
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

In Citus, we support aggregate functions by whitelisting them in planning time by ingesting proper aggregate functions into the worker and master queries separately. We process the aggregates in [MasterAggregateExpression](https://github.com/citusdata/citus/blob/b1e66363982b883d4b3be1208746b5ff99105f10/src/backend/distributed/planner/multi_logical_optimizer.c#L1500) and [WorkerAggregateExpressionList](https://github.com/citusdata/citus/blob/b1e66363982b883d4b3be1208746b5ff99105f10/src/backend/distributed/planner/multi_logical_optimizer.c#L2667).

If we inspect the code paths in these functions, we can easily see that there are a good amount of code duplications in between the `if` - `else if` blocks. For example, these two PRs are almost identical, #2186 and #2175.

We need more generic functions to produce the necessary aggregates from the given master and worker aggregate function names. One very basic and (not even) a half implemented method for master aggregate can be seen in custom_aggregate_support branch in [this line](https://github.com/citusdata/citus/blob/custom_aggregate_support/src/backend/distributed/planner/multi_logical_optimizer.c#L1785)

Contributor guide

Open the contributing guide

Research direction

Start in src/backend/distributed/planner/multi_logical_optimizer.c at MasterAggregateExpression and WorkerAggregateExpressionList, then compare the duplicated paths and PRs #2186 and #2175. Read the partial master-aggregate method in the custom_aggregate_support branch. Done means generic functions produce the required aggregates from master and worker aggregate names without duplicating the planning logic.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql, sql
Domain
backend, databases, distributed-systems
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.