apache / apache/gluten

[VL] Array with null value will fallback

Open
#6,625 5 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

```
val aai = Literal.create(Seq(Seq(1, 2, 3), null, Seq(4, 5)),
ArrayType(ArrayType(IntegerType, containsNull = false), containsNull = true))

16:32:17.693 WARN org.apache.spark.sql.execution.GlutenFallbackReporter: Validation failed for plan: Project[QueryId=0], due to: Native validation failed:
Optional[Validation failed due to exception caught at file:SubstraitToVeloxPlanValidator.cc line:1306 function:validate, thrown from file:SubstraitToVeloxExpr.cc line:400 function:literalsToArrayVector, reason:(0 vs. 0) there should be at least 1 value in list literal.].
```

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash
"literal": {
"list": {
"values": [
{
"list": {
"values": [
{
"i32": 1
},
{
"i32": 2
},
{
"i32": 3
}
]
}
},
{
"null": {
"list": {
"type": {
"i32": {
"nullability": "NULLABILITY_REQUIRED"
}
},
"nullability": "NULLABILITY_NULLABLE"
}
}
},
{
"list": {
"values": [
{
"i32": 4
},
{
"i32": 5
}
]
}
}
]
}
}
```

Contributor guide

Open the contributing guide

Research direction

Start with literalsToArrayVector in SubstraitToVeloxExpr.cc around line 400 and the validation call in SubstraitToVeloxPlanValidator.cc around line 1306. Reproduce the issue with the Literal.create array shown in the report, then verify that an array containing a null nested value no longer triggers native validation failure or fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, sql
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.