[FLINK] Support common stateless scalar expressions
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
## Motivation
The velox backend already implements a set of common Flink SQL scalar functions, but the gluten-flink planner has no mapping from Flink function name to velox Substrait operator, so these functions cannot be offloaded to velox today.
Functions in scope:
- Time fields: `YEAR`, `MONTH`, `DAYOFMONTH`, `DAYOFYEAR`, `DAYOFWEEK`, `QUARTER`, `HOUR`, `MINUTE`, `SECOND`, `WEEK`
- String/control: `SUBSTRING`, `COALESCE`, `FROM_BASE64`, `JSON_VALUE`
Additionally, `RexNodeConverter` does not convert TIMESTAMP literals to `TimestampValue.create(epochSeconds, nanos)`, so any literal-TIMESTAMP-bearing expression fails earlier in the conversion path.
## Proposed Capability
The planner should produce a valid Substrait plan for SQL like:
```sql
SELECT
EXTRACT(HOUR FROM ts),
SUBSTRING(name, 1, 3),
COALESCE(name, 'unknown'),
JSON_VALUE(payload, '$.user.id')
FROM ...
```
### Gluten version
main branch
Contributor guide
Research direction
Start by tracing the gluten-flink planner's function-name mapping into Velox Substrait operators, then read the RexNodeConverter timestamp-literal conversion path. Verify coverage for the listed time, string, control, and JSON_VALUE expressions, plus TIMESTAMP literals. Done means the example SQL produces valid Substrait plans and literal-timestamp expressions no longer fail during conversion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, sql
- Domain
- backend, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100