apache / apache/pinot

NullPointerException is thrown when query with aggregation on top of groovy functions

Open
#6,253 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

I’m getting `NPE` in `GroupByOrderByCombineOperator.getNextBlock` when the query does aggregation on top of groovy function, no matter which column I group-by with (eg dimension, datetime), or what aggregation function I use (eg AVG, SUM). The query would look like,
```sql
select
country_code,
avg(groovy('{"returnType":"DOUBLE","isSingleValue":true}', 'arg0 > arg1 ? arg0 : arg1', subtotal, total)) as average_rev
from orders
group by country_code
limit 10
```
Without `group by`, aforementioned `groovy` function works fine with enough records.

### Expected results

Expect aggreation on top of transform would work for groovy just like other transform function like `ADD` or `SUB`

### Actual results

NPE thrown as,
```log
QueryExecutionError:
java.lang.NullPointerException
at org.apache.pinot.core.operator.combine.GroupByOrderByCombineOperator.getNextBlock(GroupByOrderByCombineOperator.java:215)
at org.apache.pinot.core.operator.combine.GroupByOrderByCombineOperator.getNextBlock(GroupByOrderByCombineOperator.java:62)
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:37)
at org.apache.pinot.core.operator.InstanceResponseOperator.getNextBlock(InstanceResponseOperator.java:26)
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49)
at org.apache.pinot.core.plan.GlobalPlanImplV0.execute(GlobalPlanImplV0.java:48)
at org.apache.pinot.core.query.executor.ServerQueryExecutorV1Impl.processQuery(ServerQueryExecutorV1Impl.java:221)
at org.apache.pinot.core.query.scheduler.QueryScheduler.processQueryAndSerialize(QueryScheduler.java:155)
at org.apache.pinot.core.query.scheduler.QueryScheduler.lambda$createQueryFutureTask$0(QueryScheduler.java:139)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
at shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
```

UPDATE1:
We're running v0.5.0

Contributor guide

Open the contributing guide

Research direction

Start at GroupByOrderByCombineOperator.getNextBlock, especially line 215 in the stack trace, and reproduce the provided aggregation-over-groovy query on Pinot v0.5.0. Compare grouped aggregation with and without the groovy transform and verify that the query completes without a NullPointerException and returns the expected aggregate results.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.