Numerical type conversion of literal values in IN and NOT_IN clause
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Pinot currently supports numerical type conversion of literal values in WHERE clause as shown in the examples below:
```
select runs from baseballStats WHERE runs = 10.0
select runs from baseballStats WHERE runs = 10.3
select runs from baseballStats WHERE runs != 10.3
select runs from baseballStats WHERE runs > 6.5 AND runs < 10.23
```
This support was added through PR https://github.com/apache/pinot/pull/6811 and https://github.com/apache/pinot/pull/6927. However, Pinot still doesn't support type conversion of literal values that may appear in the IN clause. For example, the queries shown below will throw exceptions:
```
select runs from baseballStats WHERE runs IN (5.0, 6, 7.2)
select runs from baseballStats WHERE runs NOT IN (5.0, 6, 7.2)
```
We would like to extend the type conversion mechanism in `NumericalFilterOptimizer` to cover IN and NOT_IN clauses as well. One side effect of this change would be that we would need to make another pass over all the literal values specified in IN or NOT_IN clause when these clauses are present in the query.
Contributor guide
Research direction
Start with NumericalFilterOptimizer and trace how literal values are converted for existing WHERE comparisons. Reproduce the two IN and NOT IN queries from the issue, then check the relevant query optimizer tests or add coverage for mixed numeric literals. Done means both queries execute without conversion exceptions while existing comparison behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100