apache / apache/pinot

ArrayIndexOutOfBoundsException from GroupByOrderByCombineOperator

Open
#6,393 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

We're getting `QueryExecutionError:\njava.lang.ArrayIndexOutOfBoundsException` while executing below query:
```sql
SELECT name as key_col, COUNT(*) as val_col
FROM orders
WHERE created_at_seconds BETWEEN 1606756268 AND 1609175468
AND (organization_id <> 'example_uuid')
GROUP BY key_col
ORDER BY val_col DESC
LIMIT 5
```

The error log on server nodes are like,

```log
2020/12/29 17:50:16.761 ERROR [GroupByOrderByCombineOperator] [pqw-7] Caught exception while processing and combining group-by order-by for index: 1, operator:
org.apache.pinot.core.operator.query.AggregationGroupByOrderByOperator, queryContext: QueryContext{_selectExpressions=[name, count(*)], _aliasMap={name=key_col, count(*)=val_col},
_filter=(created_at_seconds BETWEEN '1606756268' AND '1609175468' AND organization_id != 'example_uuid'), _groupByExpressions=[name], _orderByExpressions=[count(*) DESC], _havingFilter=null,
_limit=5, _offset=0, _queryOptions={responseFormat=sql, groupByMode=sql, timeoutMs=24999}, _debugOptions=null, _brokerRequest=BrokerRequest(querySource:QuerySource(
tableName:enriched_station_orders_v1_OFFLINE), filterQuery:FilterQuery(id:0, value:null, operator:AND, nestedFilterQueryIds:[1, 2]), aggregationsInfo:[AggregationInfo(aggregationType:COUNT,
aggregationParams:{column=*}, isInSelectList:true, expressions:[*])], groupBy:GroupBy(topN:5, expressions:[name]), filterSubQueryMap:FilterQueryMap(filterQueryMap:{0=FilterQuery(id:0, value:null,
operator:AND, nestedFilterQueryIds:[1, 2]), 1=FilterQuery(id:1, column:created_at_seconds, value:[[1606756268 1609175468]], operator:RANGE, nestedFilterQueryIds:[]), 2=FilterQuery(id:2,
column:organization_id, value:[example_uuid], operator:NOT, nestedFilterQueryIds:[])}), queryOptions:{responseFormat=sql, groupByMode=sql, timeoutMs=24999}, pinotQuery:PinotQuery(
dataSource:DataSource(tableName:enriched_station_orders_v1_OFFLINE), selectList:[Expression(type:FUNCTION, functionCall:Function(operator:AS, operands:[Expression(type:IDENTIFIER,
identifier:Identifier(name:name)), Expression(type:IDENTIFIER, identifier:Identifier(name:key_col))])), Expression(type:FUNCTION, functionCall:Function(operator:AS, operands:[Expression(
type:FUNCTION, functionCall:Function(operator:COUNT, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:*))])), Expression(type:IDENTIFIER, identifier:Identifier(name:val_col))]))],
filterExpression:Expression(type:FUNCTION, functionCall:Function(operator:AND, operands:[Expression(type:FUNCTION, functionCall:Function(operator:BETWEEN, operands:[Expression(type:IDENTIFIER,
identifier:Identifier(name:created_at_seconds)), Expression(type:LITERAL, literal:), Expression(type:LITERAL, literal:)])), Expression(
type:FUNCTION, functionCall:Function(operator:NOT_EQUALS, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:organization_id)), Expression(type:LITERAL, literal:)]))])), groupByList:[Expression(type:IDENTIFIER, identifier:Identifier(name:name))], orderByList:[Expression(type:FUNCTION, functionCall:Function(operator:DESC,
operands:[Expression(type:FUNCTION, functionCall:Function(operator:COUNT, operands:[Expression(type:IDENTIFIER, identifier:Identifier(name:*))]))]))], limit:5), orderBy:[SelectionSort(
column:count(*), isAsc:false)], limit:5)}
```
Similar query would work fine on same table with different set of data, or other column. So this might be some kind of data hitting a issue in the logic.

Contributor guide

Open the contributing guide

Research direction

Read the GroupByOrderByCombineOperator and AggregationGroupByOrderByOperator paths named in the log, then run the supplied SQL against the affected table and data. Done means the query no longer raises ArrayIndexOutOfBoundsException and the failure has regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.