apache / apache/gluten

[VL] try_cast does not return NULL

Open
#8,108 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

```
val data = Seq(("1234567"), ("000A1234"), ("ABCDE0"))
val df = spark.createDataFrame(data.map(Tuple1(_))).toDF("value")
df.write.format("parquet").mode("overwrite").save("file:///home/a/src/sparklocal/tbl)
spark.read.format("parquet").load("file:///home/a/src/sparklocal/wrong").createOrReplaceTempView("tbl")
val df = sql("select try_cast(value as bigint) from tbl")
```
Returns
```
E1126 15:27:22.105729 44701 CastExpr-inl.h:43] Cannot cast VARCHAR '000A1234' to BIGINT. Encountered a non-digit character
I1126 15:27:22.106130 44701 Task.cpp:1879] Terminating task Gluten_Stage_2_TID_2_VTID_1 with state Failed after running for 5 ms.
I1126 15:27:22.106395 44701 Task.cpp:1136] All drivers (1) finished for task Gluten_Stage_2_TID_2_VTID_1 after running for 5 ms.
24/11/26 15:27:22 ERROR TaskResources: Task 2 failed by error: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException:
Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Cannot cast VARCHAR '000A1234' to BIGINT. Encountered a non-digit character
Retriable: False
Context: Top-level
Expression: cast((trim(
             

:VARCHAR, n0_0)) as BIGINT)
```
While Spark returns Array([1234567], [null], [null])

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Spark SQL query using Velox and inspect the cast failure reported from CastExpr-inl.h:43. Compare try_cast behavior for valid and invalid strings with Spark; done means invalid BIGINT conversions return NULL and the query completes instead of failing the task.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.