apache / apache/gluten

[VL] chartype cause projectexec fallback with spark3.4

Open
#3,833 1 comment 1 reaction 0 assignees View on GitHub
bug enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

In spark3.4, because of the patch https://issues.apache.org/jira/browse/SPARK-40697, gluten can not transform ProjectExec including chartype columns to native, maybe we should add this to Limitation.doc or recommand users to disable `spark.sql.readSideCharPadding`

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Limitation.doc and the Spark 3.4 context described in SPARK-40697, then check how the spark.sql.readSideCharPadding setting relates to the reported fallback. Done means documenting the limitation or clearly recommending that users disable the setting, with the affected chartype case explained.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.