Boolean expression on top of a MV column filter fails
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Attempting to execute a query like `SELECT * FROM table WHERE (mvCol = 'test') = true`, fails with the exception:
```
2023/03/25 01:17:47.521 ERROR [BaseCombineOperator] [pqw-2] Caught exception while processing query: QueryContext{_tableName='random_REALTIME', _subquery=null, _selectExpressions=[multiCol, timestamp], _aliasList=[null, null], _filter=equals(multiCol,'null') = 'true', _groupByExpressions=null, _havingFilter=null, _orderByExpressions=null, _limit=10, _offset=0, _queryOptions={responseFormat=sql, groupByMode=sql, timeoutMs=9999}, _expressionOverrideHints={}, _explain=false}
java.lang.UnsupportedOperationException: null
at org.apache.pinot.segment.spi.index.mutable.MutableForwardIndex.readDictIds(MutableForwardIndex.java:74) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.segment.spi.index.mutable.MutableForwardIndex.readDictIds(MutableForwardIndex.java:79) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.common.DataFetcher$ColumnValueReader.readStringValues(DataFetcher.java:570) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.common.DataFetcher.fetchStringValues(DataFetcher.java:239) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.common.DataBlockCache.getStringValuesForSVColumn(DataBlockCache.java:277) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.docvalsets.ProjectionBlockValSet.getStringValuesSV(ProjectionBlockValSet.java:153) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.transform.function.IdentifierTransformFunction.transformToStringValuesSV(IdentifierTransformFunction.java:111) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.fillResultString(BinaryOperatorTransformFunction.java:284) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.fillResultArray(BinaryOperatorTransformFunction.java:135) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.transformToIntValuesSV(BinaryOperatorTransformFunction.java:109) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.dociditerators.ExpressionScanDocIdIterator.processProjectionBlock(ExpressionScanDocIdIterator.java:156) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.dociditerators.ExpressionScanDocIdIterator.next(ExpressionScanDocIdIterator.java:88) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:75) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:39) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:43) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:70) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:37) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:43) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:97) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:41) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:43) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.combine.BaseSingleBlockCombineOperator.processSegments(BaseSingleBlockCombineOperator.java:102) ~[pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.operator.combine.BaseCombineOperator$1.runJob(BaseCombineOperator.java:107) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.core.util.trace.TraceRunnable.run(TraceRunnable.java:40) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at org.apache.pinot.shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at org.apache.pinot.shaded.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [pinot-all-0.13.0-SNAPSHOT-jar-with-dependencies.jar:0.13.0-SNAPSHOT-3f36623c563b5ad495a502a00f627b44bfc0861d]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
```
# Version
apache/pinot@3f36623
Contributor guide
Research direction
Reproduce the Boolean expression query against the Pinot version identified in the report, then trace the failure from MutableForwardIndex.readDictIds through DataFetcher and BinaryOperatorTransformFunction. Compare the MV-column filter path with the expression scan path shown in the stack trace. Done means the query executes without the UnsupportedOperationException and preserves the expected filter behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100