MomentSketchAggregatorFactory doesn't implement makeAggregateCombiner
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
In 0.23.0, using a `momentSketch` aggregator at ingestion will result in the following exception during indexing merging:
```
2022-10-12T07:35:37,617 WARN [[index_parallel_newyork_momentsketch_test_agfjijan_2022-10-12T07:35:26.205Z]-appenderator-merge] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Failed to push merged index for segment[newyork_momentsketch_test_2016-01-01T00:00:00.000Z_2016-02-01T00:00:00.000Z_2022-10-12T07:35:36.144Z].
org.apache.druid.java.util.common.UOE: [org.apache.druid.query.aggregation.momentsketch.aggregator.MomentSketchAggregatorFactory] does not implement makeAggregateCombiner()
at org.apache.druid.query.aggregation.AggregatorFactory.makeAggregateCombiner(AggregatorFactory.java:128) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.query.aggregation.AggregatorFactory.makeNullableAggregateCombiner(AggregatorFactory.java:142) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.lambda$new$1(RowCombiningTimeAndDimsIterator.java:120) ~[druid-processing-0.23.0.jar:0.23.0]
at java.util.Arrays.setAll(Arrays.java:4698) ~[?:1.8.0_275]
at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.(RowCombiningTimeAndDimsIterator.java:120) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.lambda$merge$3(IndexMergerV9.java:1298) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.makeMergedTimeAndDimsIterator(IndexMergerV9.java:1360) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:293) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:1303) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.multiphaseMerge(IndexMergerV9.java:1120) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:1062) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:923) ~[druid-server-0.23.0.jar:0.23.0]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:784) ~[druid-server-0.23.0.jar:0.23.0]
at com.google.common.util.concurrent.Futures$1.apply(Futures.java:713) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861) [guava-16.0.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
```
It looks like the moment sketch aggregator doesn't have `makeAggregateCombiner()`.
The ingestion spec:
```
...
{
"type": "momentSketch",
"k": 3,
"name": "ms_passenger_count_true",
"fieldName": "passenger_count",
"compress": true
}
...
```
Contributor guide
Research direction
Start with MomentSketchAggregatorFactory and compare its aggregation behavior with AggregatorFactory.makeAggregateCombiner(), using the stack trace to follow the ingestion merge path through RowCombiningTimeAndDimsIterator. Reproduce the momentSketch ingestion spec and confirm that indexing merge completes without the makeAggregateCombiner() exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100