[VL] When the regexExpr of the split function is an empty character or a non-ASCII character, fall back to Valina Spark
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Backend
VL (Velox)
### Bug description
**empty character**
```txt
Expression: split('abc','')
Spark: ["a","b","c",""]
Gluten: Reason: (0 vs. 1) split only supports only single-character pattern
```
**non-ASCII character**
```txt
Expression: split('a,b,c',',')
Spark: ["a","b","c"]
Gluten: Reason: (3 vs. 1) split only supports only single-character pattern
```
### Spark version
Spark-3.2.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
java.lang.RuntimeException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: (0 vs. 1) split only supports only single-character pattern
Retriable: False
Expression: patternString.size() == 1
Context: split(n0_0, :VARCHAR)
Top-Level Context: Same as context.
Function: apply
File: ep/build-velox/build/velox_ep/velox/functions/sparksql/SplitFunctions.cpp
```
Contributor guide
Research direction
Start with the Velox split entry point in SplitFunctions.cpp and trace how Gluten dispatches split expressions. Reproduce the empty-pattern and non-ASCII cases against Spark 3.2.x, then verify that unsupported patterns use the Vanilla Spark fallback and match the documented results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, spark
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100