apache / apache/gluten

[VL] When the regexExpr of the split function is an empty character or a non-ASCII character, fall back to Valina Spark

Open
#6,127 5 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

**empty character**
```txt
Expression: split('abc','')
Spark: ["a","b","c",""]
Gluten: Reason: (0 vs. 1) split only supports only single-character pattern
```

**non-ASCII character**
```txt
Expression: split('a,b,c',',')
Spark: ["a","b","c"]
Gluten: Reason: (3 vs. 1) split only supports only single-character pattern
```

### Spark version

Spark-3.2.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash
java.lang.RuntimeException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: (0 vs. 1) split only supports only single-character pattern
Retriable: False
Expression: patternString.size() == 1
Context: split(n0_0, :VARCHAR)
Top-Level Context: Same as context.
Function: apply
File: ep/build-velox/build/velox_ep/velox/functions/sparksql/SplitFunctions.cpp
```

Contributor guide

Open the contributing guide

Research direction

Start with the Velox split entry point in SplitFunctions.cpp and trace how Gluten dispatches split expressions. Reproduce the empty-pattern and non-ASCII cases against Spark 3.2.x, then verify that unsupported patterns use the Vanilla Spark fallback and match the documented results.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, spark
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.