[CORE] Bad mutable query plan nodes
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Description
Spark / Gluten query planner requires heavily on plan immutability.
Currently there are some (unlimited to) code snippets potentially making plans mutable:
1. WholeStageTransformer.setOutputSchemaForPlan (CH) https://github.com/apache/incubator-gluten/blob/15f4cde02bf965e70bae5079c40aae78ad80e45a/gluten-substrait/src/main/scala/org/apache/gluten/execution/WholeStageTransformer.scala#L234-L241
2. BatchScanExecTransformer.setPushDownFilters (VL + CH) https://github.com/apache/incubator-gluten/blob/15f4cde02bf965e70bae5079c40aae78ad80e45a/gluten-substrait/src/main/scala/org/apache/gluten/execution/BatchScanExecTransformer.scala#L102-L104
Should find a way to optimize against the code to make relevant plan immutable.
Contributor guide
Research direction
Start by reading WholeStageTransformer.setOutputSchemaForPlan and BatchScanExecTransformer.setPushDownFilters at the linked revisions. Trace how these methods affect Spark/Gluten query plan nodes for the CH, VL, and CH paths. Done means the relevant plan nodes remain immutable while preserving the required planner behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100