apache / apache/gluten

[VL][CH] Discrepancy in file path between CH and VL

Open
#10,805 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

CH encodes file paths into ASCII in `BackendsApiManager.getTransformerApiInstance.encodeFilePathIfNeed` and `CHIteratorApi.genSplitInfo` whereas VL doesn't do that. Why is there such difference between the two engines? Is this because CH is not able to work with non-ASCII file paths?

### Gluten version

_No response_

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Research direction

Compare BackendsApiManager.getTransformerApiInstance.encodeFilePathIfNeed with CHIteratorApi.genSplitInfo, focusing on how each engine handles non-ASCII paths. Establish whether ClickHouse requires ASCII encoding, then document the reason or define the needed behavior and verify it with a representative non-ASCII file path.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, scala
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.