Complete the removal of aggregators from Spark runner.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
In https://issues.apache.org/jira/browse/BEAM-1765 aggregator usage was removed from Spark runner in order to comply with the removal of aggregators from the SDK.
However, not all classes / usages of aggregators within the Spark runner were removed.
This task is to complete the removal of all aggregator code from Spark runner, including, but not limited to:
```
org.apache.beam.runners.spark.aggregators.AggAccumParam
org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator
org.apache.beam.runners.spark.aggregators.NamedAggregators
org.apache.beam.runners.spark.metrics.AggregatorMetricSource
org.apache.beam.runners.spark.metrics.AggregatorMetric
```
In addition, remove packages named `aggregators` and move any remaining relevant classes to `org.apache.beam.runners.spark.metrics` package.
Imported from Jira [BEAM-2811](https://issues.apache.org/jira/browse/BEAM-2811). Original Jira may contain additional context.
Reported by: aviemzur.
Contributor guide
Research direction
Start by searching the Spark runner for the listed classes and all references to aggregator code, including the aggregators and metrics packages. Read the BEAM-1765 and BEAM-2811 context before deciding which remaining classes belong under org.apache.beam.runners.spark.metrics. Done means aggregator usages and the aggregators packages are removed, with relevant classes moved to the metrics package.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100