apache / apache/gluten

[VL]Validate GenerateRel not attempt to compile the generator.

Open
#6,574 3 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Validate GenerateRel did not attempt to compile the generator. If the generator is not supported, Spark SQL will not fallback to GenerateRel, which will result in job failure.
For example v1.1.1, sql:
```
SELECT explode(slice(a, 2, 2)) from t
```
spark job willl failed
```
Job aborted due to stage failure: Task 1 in stage 26.0 failed 4 times, most recent failure: Lost task 1.3 in stage 26.0 (TID 360) (xx executor 11): io.glutenproject.exception.GlutenException: java.lang.RuntimeException: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Scalar function slice not registered with arguments: (ARRAY, INTEGER, INTEGER). Found function registered with the following signatures:
((array(T),bigint,bigint) -> array(T))
Retriable: False
Function: compileRewrittenExpression
File: /root/package/gluten/ep/build-velox/build/velox_ep/velox/expression/ExprCompiler.cpp
Line: 468
```

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.