apache / apache/gluten

[VL] Improve scan only mode to include filter node

Open
#10,392 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Description

Currently the filter is already included in the scan node of "scan_only" mode. We could eliminate the filter node in vanilla Spark as Velox already did the filter in scan.

Cc @rui-mo

### Gluten version

None

Contributor guide

Open the contributing guide

Research direction

Start by tracing the scan_only mode's scan node and filter node handling, then compare how vanilla Spark and Velox apply the filter. Done means the redundant filter node is eliminated for vanilla Spark while scan behavior remains correct; the issue names no files or tests to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
data
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.