NullPointerException is thrown when query with aggregation on top of groovy functions
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
I’m getting `NPE` in `GroupByOrderByCombineOperator.getNextBlock` when the query does aggregation on top of groovy function, no matter which column I group-by with (eg dimension, datetime), or what aggregation function I use (eg AVG, SUM). The query would look like,
```sql
select
country_code,
avg(groovy('{"returnType":"DOUBLE","isSingleValue":true}', 'arg0 > arg1 ? arg0 : arg1', subtotal, total)) as average_rev
from orders
group by country_code
limit 10
```
Without `group by`, aforementioned `groovy` function works fine with enough records.
### Expected results
Expect aggreation on top of transform would work for groovy just like other transform function like `ADD` or `SUB`
### Actual results
NPE thrown as,
```log
QueryExecutionError:
java.lang.NullPointerException
at org.apache.pinot.core.operator.combine.GroupByOrderByCombineOperator.getNextBlock(GroupByOrderByCombineOperator.java:215)
at org.apache.pinot.core.operator.combine.GroupByOrderByCombineOperator.getNextBlock(GroupByOrderByCombineOperator.java:62)
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37)
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26)
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48)
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:221)
at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:155)
at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:139)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
at shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
```
UPDATE1:
We're running v0.5.0
Contributor guide
Research direction
Start at GroupByOrderByCombineOperator.getNextBlock, especially line 215 in the stack trace, and reproduce the provided aggregation-over-groovy query on Pinot v0.5.0. Compare grouped aggregation with and without the groovy transform and verify that the query completes without a NullPointerException and returns the expected aggregate results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy, java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100