apache / apache/gluten

[VL] Pass the table schema to the HiveTableHandle

Open
#10,568 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Today we pass the columns we expect to read from the table as the "dataColumns" parameter of the HiveTableHandle. What HiveTableHandle expects to receive here is the schema of the table being read. This mostly works today because for the file formats we support (Parquet and ORC) we hard code the configs to force them to map the file columns to the table columns by name, so the missing columns aren't needed. However, in order to support mapping file columns to table columns by index (an option in Velox) or other file formats that are index based (e.g. Text) we need to pass the table schema. This shouldn't affect the current behavior as this is just a superset of what we're currently using.

### Gluten version

None

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.