apache / apache/druid

Broker throws RuntimeException on GroupBy metric with filter applied on NaN values

Open
#11,623 2 comments 0 reactions 0 assignees View on GitHub
Area - SQL
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version
0.19

The Druid version where the problem was encountered.
0.19

### Description

Simple groupby query like

```
SELECT sum(someMetric), someDimension from someDatasource
WHERE someMetric <> 'NaN'
GROUP BY someDimension
```

fails with on the Query tab:
Error: Unknown exception

While invoking method 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'

java.lang.RuntimeException

stacktrace:

```
java.lang.RuntimeException: While invoking method 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:272) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:161) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.druid.sql.calcite.planner.Rules$DecorrelateAndTrimFieldsProgram.run(Rules.java:301) ~[druid-sql-0.20.0.jar:0.20.0]
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:357) ~[calcite-core-1.21.0.jar:1.21.0]
at org.apache.druid.sql.calcite.planner.DruidPlanner.planWithDruidConvention(DruidPlanner.java:203) ~[druid-sql-0.20.0.jar:0.20.0]
at org.apache.druid.sql.calcite.planner.DruidPlanner.plan(DruidPlanner.java:137) ~[druid-sql-0.20.0.jar:0.20.0]
at org.apache.druid.sql.SqlLifecycle.plan(SqlLifecycle.java:168) ~[druid-sql-0.20.0.jar:0.20.0]
at org.apache.druid.sql.SqlLifecycle.plan(SqlLifecycle.java:179) ~[druid-sql-0.20.0.jar:0.20.0]
....
....
Caused by: java.lang.NumberFormatException
at java.math.BigDecimal.(BigDecimal.java:497) ~[?:1.8.0_302]
at java.math.BigDecimal.(BigDecimal.java:383) ~[?:1.8.0_302]
at java.math.BigDecimal.(BigDecimal.java:809) ~[?:1.8.0_302]
at java.math.BigDecimal.valueOf(BigDecimal.java:1277) ~[?:1.8.0_302]
at org.apache.druid.sql.calcite.planner.DruidRexExecutor.reduce(DruidRexExecutor.java:129) ~[druid-sql-0.20.0.jar:0.20.0]

```

Contributor guide

Open the contributing guide

Research direction

Reproduce the SQL query using a GroupBy metric filtered against NaN, then inspect org.apache.druid.sql.calcite.planner.DruidRexExecutor.reduce and the planner path shown in the stack trace. Done means the query no longer throws RuntimeException or NumberFormatException during planning and returns the expected grouped result.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.