apache / apache/gluten

[VL] Column name containing parts of Cyrillic cannot be read correctly

Open
#6,843 8 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

```
import testImplicits._
Seq((1, 2)).toDF("Товары", "овары").write.mode("overwrite").parquet("tmp/t1")
spark.read.parquet("tmp/t1").show()
spark.conf.set("spark.gluten.enabled", false)
spark.read.parquet("tmp/t1").show()
```
enable gluten
```
+------+-----+
|Товары|овары|
+------+-----+
| null| 2|
+------+-----+
```
disable gluten
```
+------+-----+
|Товары|овары|
+------+-----+
| 1| 2|
+------+-----+
```

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.