apache / apache/druid

SQL query returns StackOverFlowError

Open
#9,932 0 comments 0 reactions 0 assignees View on GitHub
Area - SQL Bug
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

SQL query returns StackOverFlowError

### Affected Version

0.18.1

### Description

Druid SQL queries sometimes would return StackOverflowError like the following example:

```
SELECT STDDEV( ( lo_partkey ) ) FILTER ( WHERE ( ( lo_orderdate ) < ( lo_linenumber * lo_partkey ) ) ) AS C1
FROM ssb1x
WHERE ( ( __time ) <> ( __time ) )
GROUP BY CUBE ( lo_orderkey )
HAVING COUNT( ( lo_orderkey ) ) != 0
ORDER BY 1
LIMIT 100

javax.servlet.ServletException: java.lang.StackOverflowError
```

The stack trace from the broker log shows:

```
Caused by: java.lang.StackOverflowError
at org.apache.calcite.rel.core.Values.estimateRowCount(Values.java:170) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:210) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:70) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:131) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:70) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:75) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:266) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_DistinctRowCount.getDistinctRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:774) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:194) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:70) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdUtil.estimateFilteredRows(RelMdUtil.java:750) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:122) ~[calcite-core-1.21.0.jar:1.21.0]
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) ~[?:?]
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) ~[?:?]
at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225) ~[calcite-core-1.21.0.jar:1.21.0]

....
```

This may be related to https://issues.apache.org/jira/browse/CALCITE-2057, but we don't know for sure until further investigation is done.

Contributor guide

Open the contributing guide

Research direction

Reproduce the SQL query against Druid 0.18.1 and inspect the broker stack trace around Calcite's RelMdRowCount, RelMdDistinctRowCount, RelMetadataQuery, and Values. Determine why metadata estimation recurses for this query and verify that the query no longer produces StackOverflowError.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.