apache / apache/druid

MomentSketchAggregatorFactory doesn't implement makeAggregateCombiner

Open
#13,220 2 comments 0 reactions 0 assignees View on GitHub
Bug
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

In 0.23.0, using a `momentSketch` aggregator at ingestion will result in the following exception during indexing merging:

```
2022-10-12T07:35:37,617 WARN [[index_parallel_newyork_momentsketch_test_agfjijan_2022-10-12T07:35:26.205Z]-appenderator-merge] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Failed to push merged index for segment[newyork_momentsketch_test_2016-01-01T00:00:00.000Z_2016-02-01T00:00:00.000Z_2022-10-12T07:35:36.144Z].
org.apache.druid.java.util.common.UOE: [org.apache.druid.query.aggregation.momentsketch.aggregator.MomentSketchAggregatorFactory] does not implement makeAggregateCombiner()
at org.apache.druid.query.aggregation.AggregatorFactory.makeAggregateCombiner(AggregatorFactory.java:128) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.query.aggregation.AggregatorFactory.makeNullableAggregateCombiner(AggregatorFactory.java:142) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.lambda$new$1(RowCombiningTimeAndDimsIterator.java:120) ~[druid-processing-0.23.0.jar:0.23.0]
at java.util.Arrays.setAll(Arrays.java:4698) ~[?:1.8.0_275]
at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.(RowCombiningTimeAndDimsIterator.java:120) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.lambda$merge$3(IndexMergerV9.java:1298) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.makeMergedTimeAndDimsIterator(IndexMergerV9.java:1360) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:293) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:1303) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.multiphaseMerge(IndexMergerV9.java:1120) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:1062) ~[druid-processing-0.23.0.jar:0.23.0]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.mergeAndPush(AppenderatorImpl.java:923) ~[druid-server-0.23.0.jar:0.23.0]
at org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.lambda$push$1(AppenderatorImpl.java:784) ~[druid-server-0.23.0.jar:0.23.0]
at com.google.common.util.concurrent.Futures$1.apply(Futures.java:713) [guava-16.0.1.jar:?]
at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:861) [guava-16.0.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
```

It looks like the moment sketch aggregator doesn't have `makeAggregateCombiner()`.

The ingestion spec:
```
...
{
"type": "momentSketch",
"k": 3,
"name": "ms_passenger_count_true",
"fieldName": "passenger_count",
"compress": true
}
...
```

Contributor guide

Open the contributing guide

Research direction

Start with MomentSketchAggregatorFactory and compare its aggregation behavior with AggregatorFactory.makeAggregateCombiner(), using the stack trace to follow the ingestion merge path through RowCombiningTimeAndDimsIterator. Reproduce the momentSketch ingestion spec and confirm that indexing merge completes without the makeAggregateCombiner() exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.