[Feature] support type inference and automatic type casting when searching for ScalarFunction
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Background
===
Currently Pinot already implements some of the type hoisting - for example:
* the datatable builder automatically converts int/float/long/double numeric type content to the corresponding data schema required data type.
However sometime this doesn't work quite well. for example
* sum(long, long) will result in double
Propose
===
- Create a type inference system to automatically hoist types to when searching for a matching `@ScalarFunction`. for example `sum(int, long)` should match `sum(long, long)` with an implicit type cast from `int` to `long` for the first argument.
- implementing a type casting scheme using a bitmap approach:
- for example using numeric values, `double, float, long, int, short, byte` corresponding to a single bit in a byte value (`1<<6` ~ `1<<0`) and make all function arguments to mask into the byte and uses the highest non-zero bit corresponding type as the numeric type to cast to.
- for other type we can have different rules to match types
- implicit type cast methods can also be unified and reused in other part of the query logic and ingestion.
Contributor guide
Research direction
Start by examining the existing ScalarFunction lookup and the datatable builder's numeric type handling. Define how type inference should match calls such as sum(int, long) to sum(long, long), including implicit casts and rules for non-numeric types. Done means matching functions can apply the intended casts and the casting approach can be reused in query logic and ingestion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100