Legacy Parser Policy doesn't exist for Velox
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
In Spark, there is legacy time parser policy(spark.sql.legacy.timeParserPolicy).
- If LEGACY, SimpleDateFormat is used for formatting and parsing dates/timestamps in a locale-sensitive manner
- If CORRECTED, classes from java.time.* packages are used for the same purpose.
- EXCEPTION, RuntimeException is thrown when we will get different results.
**Repro**:
After adding cast(timestamp -> date),
test("test to_date") {
runQueryAndCompare("select to_date('1965-01-01 10:11:12.123456','yyyy-MM-dd') as ts") { _ => }
}
This unit test fails because `spark.sql.legacy.timeParserPolicy` is configured to `exception`, which Spark adheres to. However, in Velox, the equivalent Legacy Parser Policy is absent, causing it to return null instead.
### Spark version
Spark-3.4.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.