apache / apache/gluten

[VL] Support native Avro scan

Open
#11,178 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Adds a Velox-native Avro scan operator, significantly reducing serialization overhead.

Production workloads demonstrate **~80% improvement** in scan performance.

Will submit a Draft PR shortly and iterate on the implementation, work toward merging into Gluten and Velox main.

Before:
Image

After:
Image

### Gluten version

None

Contributor guide

Open the contributing guide

Research direction

The issue proposes a Velox-native Avro scan operator but names no files, tests, or entry points. Start by locating existing scan operators in Gluten and Velox, then determine the integration and validation needed for the proposed implementation to be submitted and merged into both projects.

Written by the indexing model from the issue text.

Assessment

Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.