apache / apache/gluten

Condition eliminate between scan and filter haven't applied on iceberg/delta

Open
#5,252 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Backend

VL (Velox)

### Bug description

There is an optimization that eliminate common expression between scan and filter, but it doesn't apply on IcebergScanTransformer/DeltaScanTransformer as they are not inherited from BasicScanExecTransformer.

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by comparing IcebergScanTransformer and DeltaScanTransformer with BasicScanExecTransformer, then trace where the scan-and-filter common-expression elimination is applied. Reproduce the behavior with Iceberg and Delta scan/filter cases using the VL backend; done means the optimization applies to both transformers and is covered by relevant tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.