apache / apache/gluten

[VL] refactor CSV feature to use Velox CSV reader

Open
#9,985 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Description

Initially Gluten is using Arrow CSV reader to implement the CSV reading feature. Velox now has its own CSV reader ready. We may probably switch to use Velox CSV reader
https://github.com/facebookincubator/velox/pull/13741

### Gluten version

None

Contributor guide

Open the contributing guide

Research direction

Start by tracing Gluten’s CSV reading path and compare its current Arrow-based implementation with Velox pull request 13741. Confirm how the available Velox CSV reader can replace the existing reader; done means the CSV reading feature uses Velox instead of Arrow without changing its expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.