[VL] Improve scan only mode to include filter node
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Description
Currently the filter is already included in the scan node of "scan_only" mode. We could eliminate the filter node in vanilla Spark as Velox already did the filter in scan.
Cc @rui-mo
### Gluten version
None
Contributor guide
Research direction
Start by tracing the scan_only mode's scan node and filter node handling, then compare how vanilla Spark and Velox apply the filter. Done means the redundant filter node is eliminated for vanilla Spark while scan behavior remains correct; the issue names no files or tests to run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100