apache / apache/gluten

to_date result incorrect when the number of digit is smaller than 4

Open
#4,519 1 comment 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

When using to_date for string-to-date, the result shows difference compared to Vanilla Spark.

Below is the reproduce step:
create table to_date as select '1609304400' as t1, '1609304054' as t2 union select '1609304400' as t1, '0' as t2;
select to_date(cast(t2 as date)),t2 from to_date;

### Spark version

Spark-3.2.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by running the reported CREATE TABLE and SELECT reproduction with Spark 3.2.x and compare the results. Trace the to_date string-to-date handling in the Velox backend; done means the outputs for both provided t2 values match vanilla Spark.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark, sql
Domain
backend, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.