ArrayIndexOutOfBoundsException from GroupByOrderByCombineOperator
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
We're getting `QueryExecutionError:\njava.lang.ArrayIndexOutOfBoundsException` while executing below query:
```sql
SELECT name as key_col, COUNT(*) as val_col
FROM orders
WHERE created_at_seconds BETWEEN 1606756268 AND 1609175468
AND (organization_id <> 'example_uuid')
GROUP BY key_col
ORDER BY val_col DESC
LIMIT 5
```
The error log on server nodes are like,
```log
2020/12/29 17:50:16.761 ERROR [GroupByOrderByCombineOperator] [pqw-7] Caught exception while processing and combining group-by order-by for index: 1, operator:
org.apache.pinot.core.operator.query.AggregationGroupByOrderByOperator, queryContext: QueryContext{_selectExpressions=[name, count(*)], _aliasMap={name=key_col, count(*)=val_col},
_filter=(created_at_seconds BETWEEN '1606756268' AND '1609175468' AND organization_id != 'example_uuid'), _groupByExpressions=[name], _orderByExpressions=[count(*) DESC], _havingFilter=null,
_limit=5, _offset=0, _queryOptions={responseFormat=sql, groupByMode=sql, timeoutMs=24999}, _debugOptions=null, _brokerRequest=BrokerRequest(querySource:QuerySource(
tableName:enriched_station_orders_v1_OFFLINE), filterQuery:FilterQuery(id:0, value:null, operator:AND, nestedFilterQueryIds:[1, 2]), aggregationsInfo:[AggregationInfo(aggregationType:COUNT,
aggregationParams:{column=*}, isInSelectList:true, expressions:[*])], groupBy:GroupBy(topN:5, expressions:[name]), filterSubQueryMap:FilterQueryMap(filterQueryMap:{0=FilterQuery(id:0, value:null,
operator:AND, nestedFilterQueryIds:[1, 2]), 1=FilterQuery(id:1, column:created_at_seconds, value:[[1606756268 1609175468]], operator:RANGE, nestedFilterQueryIds:[]), 2=FilterQuery(id:2,
column:organization_id, value:[example_uuid], operator:NOT, nestedFilterQueryIds:[])}), queryOptions:{responseFormat=sql, groupByMode=sql, timeoutMs=24999}, pinotQuery:PinotQuery(
dataSource:DataSource(tableName:enriched_station_orders_v1_OFFLINE), selectList:[Expression(type:FUNCTION, functionCall:Function(operator:AS, operands:[Expression(type:IDENTIFIER,
identifier:Identifier(name:name)), Expression(type:IDENTIFIER, identifier:Identifier(name:key_col))])), Expression(type:FUNCTION, functionCall:Function(operator:AS, operands:[Expression(
type:FUNCTION, functionCall:Function(operator:COUNT, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:*))])), Expression(type:IDENTIFIER, identifier:Identifier(name:val_col))]))],
filterExpression:Expression(type:FUNCTION, functionCall:Function(operator:AND, operands:[Expression(type:FUNCTION, functionCall:Function(operator:BETWEEN, operands:[Expression(type:IDENTIFIER,
identifier:Identifier(name:created_at_seconds)), Expression(type:LITERAL, literal:), Expression(type:LITERAL, literal:)])), Expression(
type:FUNCTION, functionCall:Function(operator:NOT_EQUALS, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:organization_id)), Expression(type:LITERAL, literal:)]))])), groupByList:[Expression(type:IDENTIFIER, identifier:Identifier(name:name))], orderByList:[Expression(type:FUNCTION, functionCall:Function(operator:DESC,
operands:[Expression(type:FUNCTION, functionCall:Function(operator:COUNT, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:*))]))]))], limit:5), orderBy:[SelectionSort(
column:count(*), isAsc:false)], limit:5)}
```
Similar query would work fine on same table with different set of data, or other column. So this might be some kind of data hitting a issue in the logic.
Contributor guide
Research direction
Read the GroupByOrderByCombineOperator and AggregationGroupByOrderByOperator paths named in the log, then run the supplied SQL against the affected table and data. Done means the query no longer raises ArrayIndexOutOfBoundsException and the failure has regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100