add unit test coverage for DistributedAggregation
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
Add a dedicated `DistributedAggregationSpec.scala` that pins the four-function distributed-aggregation contract end-to-end using a representative aggregation (e.g. average): local partials computed per "node" and then merged must equal a single-node fold.
## Background
`DistributedAggregation[P <: AnyRef]` (`operator/aggregate/DistributedAggregation.scala`) is the case class that defines how an aggregate is computed in a data-parallel engine, via four functions (pattern from the SOSP'09 *Distributed Aggregation* paper). It has no dedicated unit-spec.
```scala
case class DistributedAggregation[P <: AnyRef](
init: () => P, // initial partial
iterate: (P, Tuple) => P, // accumulate one input tuple
merge: (P, P) => P, // combine two partials
finalAgg: (P) => Object // partial -> final value
)
```
## Behavior to pin
Define a representative average aggregation `DistributedAggregation[(Double, Long)]` over a single numeric column and assert:
| Step | Contract |
| --- | --- |
| `init()` | returns the identity partial `(0.0, 0L)` |
| `iterate` | folds a tuple's value in: `(sum + v, count + 1)` |
| `merge` | combines two partials additively; commutative/associative |
| `finalAgg` | `(sum, count) => sum / count` |
| distributed == single-node | split the input tuples across two partitions, `iterate` each locally from `init()`, `merge` the partials, `finalAgg` → equals the average from folding all tuples in one partition |
| empty partition | a partition with no tuples contributes `init()` and leaves the merged result unchanged |
Build `Tuple`s with the `Schema` / `Attribute` / `Tuple` helpers — see `AggregateOpSpec` in the same package for the pattern.
## Scope
- New spec: `DistributedAggregationSpec.scala` under `common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/aggregate/`.
- The spec supplies its own representative aggregation functions — the goal is to pin the case class's contract/wiring, not any specific operator.
- No production-code changes.
### Task Type
- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [x] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
Contributor guide
Assessment
This issue has not been assessed yet.