apache / apache/pinot

[multistage] Query with Anti Semi-Join Fails

Open
#10,628 7 comments 0 reactions 0 assignees View on GitHub
bug multi-stage
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

This query:

```
SELECT COUNT(*) FROM userAttributes_OFFLINE
WHERE daysSinceFirstTrip NOT IN (SELECT daysSinceFirstTrip FROM userAttributes_OFFLINE)
```

fails with:

```
2023/04/17 15:08:16.508 ERROR [OpChainSchedulerService] [query_worker_on_60908_port-7-thread-5] (OpChain{8_0_1}): Completed erroneously (8_0_1) Queued Count: 2, Executing Time: 21ms, Queued Time: 0ms {1000=class java.lang.Boolean cannot be cast to class java.lang.Number (java.lang.Boolean and java.lang.Number are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.lang.Boolean cannot be cast to class java.lang.Number (java.lang.Boolean and java.lang.Number are in module java.base of loader 'bootstrap')
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils.mergeMin(AggregationUtils.java:62)
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils$Accumulator.lambda$static$6(AggregationUtils.java:116)
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils$Accumulator.accumulate(AggregationUtils.java:174)
at org.apache.pinot.query.runtime.operator.AggregateOperator.consumeInputBlocks(AggregateOperator.java:205)}
```

Calcite defines an ordering between boolean values (see `SqlFunctions::compare`) which is the same as `Boolean.compareTo`, i.e. false is less than true. I think we need to update AggregationUtils to add support for a boolean merger for min/max functions.

@somandal : Can you lmk your thoughts?

Edit: I think the issue can happen for all anti semi-join queries, if the the input to the aggregation doesn't have unique values already and hits the else block in `AggregationUtils::accumulate`

![image](https://user-images.githubusercontent.com/8644710/232625109-1cd0dc2a-5c1b-4f76-ab82-739fd22db36f.png)

Contributor guide

Open the contributing guide

Research direction

Start in org.apache.pinot.query.runtime.operator.utils.AggregationUtils, especially mergeMin and the Accumulator.accumulate path where the Boolean is cast to Number. Reproduce the anti semi-join query and verify that min/max aggregation accepts boolean values without the ClassCastException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.