apache / apache/gluten

Errors out on timestamp column projection

Open
#10,003 4 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

While running query using velox with spark

**Result from vanilla spark:**
spark-sql (dp)> select c1 from timestamp_13_col_20_rows; 2018-05-02 09:35:09 2014-11-25 02:42:45 2017-09-23 09:42:51 2015-05-16 13:59:05 2018-05-24 09:42:10 2016-05-06 02:38:27 2018-06-04 15:11:23 2018-11-20 04:02:15 2018-09-11 05:50:33 2018-11-09 01:04:58 2015-08-11 11:41:38 2015-10-18 11:11:46 2016-04-18 15:59:06 2017-07-19 22:17:25 2018-01-19 10:05:33 2016-11-02 00:31:55 2016-01-05 23:20:15 2018-12-03 17:15:12 2016-02-06 12:25:41 2016-03-20 07:21:43 Time taken: 0.471 seconds, Fetched 20 row(s)

**

[velox_gluten_error.log](https://github.com/user-attachments/files/20791447/velox_gluten_error.log)

**
attached error log file below

**Table details:**
```spark-sql (dpe2e)> describe formatted timestamp_13_col_20_rows;
c1 timestamp
c2 timestamp
c3 timestamp
c4 timestamp
c5 timestamp
c6 timestamp
c7 timestamp
c8 timestamp
c9 timestamp
c10 timestamp
c11 timestamp
c12 timestamp

# Detailed Table Information
Catalog spark_catalog
Database dpe2e
Table timestamp_13_col_20_rows
Owner ab
Created Time Thu Apr 17 07:05:41 UTC 2025
Last Access UNKNOWN
Created By Spark 3.5.1
Type EXTERNAL
Provider parquet
Statistics 11659 bytes
Location gs://columns/timestamp_13_col_20_rows
Serde Library org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
InputFormat org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Time taken: 0.186 seconds, Fetched 28 row(s)```

**Additional Info:**
- The error seems to happen during timestamp processing.
- I originally opened this issue in the Velox repository, and was advised to report it here as it likely originates from Gluten.

### Gluten version

_No response_

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the `select c1 from timestamp_13_col_20_rows` query with Spark 3.5.x and the Velox backend, then inspect the attached `velox_gluten_error.log`. Compare the result with vanilla Spark and trace timestamp projection handling; done means the query succeeds and returns the expected timestamp values.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.