Errors out on timestamp column projection
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
While running query using velox with spark
**Result from vanilla spark:**
spark-sql (dp)> select c1 from timestamp_13_col_20_rows; 2018-05-02 09:35:09 2014-11-25 02:42:45 2017-09-23 09:42:51 2015-05-16 13:59:05 2018-05-24 09:42:10 2016-05-06 02:38:27 2018-06-04 15:11:23 2018-11-20 04:02:15 2018-09-11 05:50:33 2018-11-09 01:04:58 2015-08-11 11:41:38 2015-10-18 11:11:46 2016-04-18 15:59:06 2017-07-19 22:17:25 2018-01-19 10:05:33 2016-11-02 00:31:55 2016-01-05 23:20:15 2018-12-03 17:15:12 2016-02-06 12:25:41 2016-03-20 07:21:43 Time taken: 0.471 seconds, Fetched 20 row(s)
**
[velox_gluten_error.log](https://github.com/user-attachments/files/20791447/velox_gluten_error.log)
**
attached error log file below
**Table details:**
```spark-sql (dpe2e)> describe formatted timestamp_13_col_20_rows;
c1 timestamp
c2 timestamp
c3 timestamp
c4 timestamp
c5 timestamp
c6 timestamp
c7 timestamp
c8 timestamp
c9 timestamp
c10 timestamp
c11 timestamp
c12 timestamp
# Detailed Table Information
Catalog spark_catalog
Database dpe2e
Table timestamp_13_col_20_rows
Owner ab
Created Time Thu Apr 17 07:05:41 UTC 2025
Last Access UNKNOWN
Created By Spark 3.5.1
Type EXTERNAL
Provider parquet
Statistics 11659 bytes
Location gs://columns/timestamp_13_col_20_rows
Serde Library org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
InputFormat org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Time taken: 0.186 seconds, Fetched 28 row(s)```
**Additional Info:**
- The error seems to happen during timestamp processing.
- I originally opened this issue in the Velox repository, and was advised to report it here as it likely originates from Gluten.
### Gluten version
_No response_
### Spark version
Spark-3.5.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start by reproducing the `select c1 from timestamp_13_col_20_rows` query with Spark 3.5.x and the Velox backend, then inspect the attached `velox_gluten_error.log`. Compare the result with vanilla Spark and trace timestamp projection handling; done means the query succeeds and returns the expected timestamp values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- spark
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100