[multistage] Query with Anti Semi-Join Fails
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
This query:
```
SELECT COUNT(*) FROM userAttributes_OFFLINE
WHERE daysSinceFirstTrip NOT IN (SELECT daysSinceFirstTrip FROM userAttributes_OFFLINE)
```
fails with:
```
2023/04/17 15:08:16.508 ERROR [OpChainSchedulerService] [query_worker_on_60908_port-7-thread-5] (OpChain{8_0_1}): Completed erroneously (8_0_1) Queued Count: 2, Executing Time: 21ms, Queued Time: 0ms {1000=class java.lang.Boolean cannot be cast to class java.lang.Number (java.lang.Boolean and java.lang.Number are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.lang.Boolean cannot be cast to class java.lang.Number (java.lang.Boolean and java.lang.Number are in module java.base of loader 'bootstrap')
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils.mergeMin(AggregationUtils.java:62)
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils$Accumulator.lambda$static$6(AggregationUtils.java:116)
at org.apache.pinot.query.runtime.operator.utils.AggregationUtils$Accumulator.accumulate(AggregationUtils.java:174)
at org.apache.pinot.query.runtime.operator.AggregateOperator.consumeInputBlocks(AggregateOperator.java:205)}
```
Calcite defines an ordering between boolean values (see `SqlFunctions::compare`) which is the same as `Boolean.compareTo`, i.e. false is less than true. I think we need to update AggregationUtils to add support for a boolean merger for min/max functions.
@somandal : Can you lmk your thoughts?
Edit: I think the issue can happen for all anti semi-join queries, if the the input to the aggregation doesn't have unique values already and hits the else block in `AggregationUtils::accumulate`

Contributor guide
Research direction
Start in org.apache.pinot.query.runtime.operator.utils.AggregationUtils, especially mergeMin and the Accumulator.accumulate path where the Boolean is cast to Number. Reproduce the anti semi-join query and verify that min/max aggregation accepts boolean values without the ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100