apache / apache/gluten

[CH] The substring function is inconsistent with the Spark result

Open
#5,025 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

CH (ClickHouse)

### Bug description

SQL:select i, substring('Spark', 1, i) from range(-10, 10) t(i) where i = -3;
Spark SQL(disable native) results:
![image](https://github.com/apache/incubator-gluten/assets/37786352/bfdfdb73-04b5-4063-8ee5-98815e60b1f9)

Enable native results::
![image](https://github.com/apache/incubator-gluten/assets/37786352/622252e5-5d42-43f3-9420-b2d45381921b)

problem analysis:
Spark Function: substring
substring(str, pos[, len]) - Returns the substring of `str` that starts at `pos` and is of length `len`, or the slice of byte array that starts at `pos` and is of length `len`.

if `len` is less or equal than 0 the result is an empty string

### Spark version

Spark-3.3.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by running the reported SQL query against the ClickHouse backend and comparing its native result with Spark SQL 3.3.x. Trace the substring implementation or execution entry point used by this query; done means a non-positive length returns an empty string and the native result matches Spark.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, scala, spark
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.