Avoid rerunning logical optimizer rules on unchanged plans
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the problem
The logical optimizer reruns every rule on each optimizer pass, even when a rule already returned `Transformed::no` for the current logical plan and no later rule has changed that plan before the next invocation of the same rule.
This adds avoidable planning cost on queries that need multiple optimizer passes. ClickBench queries are a good example: many converge only after a second pass, but most rules in that pass see the exact same input they already reported as unchanged.
### Proposed improvement
Track a cheap logical plan version during optimization. Increment the version whenever a rule reports `transformed = true`. If a rule previously returned `Transformed::no` at the current plan version, skip rerunning it until some rule changes the plan.
This keeps the existing optimizer fixed-point behavior while avoiding repeated no-op work for unchanged rule inputs.
### Local measurement
Using a temporary local ClickBench logical planning probe over all 51 ClickBench queries, 30 iterations:
- Baseline optimizer-only: 607.259 us/query
- Patched optimizer-only: 409.473 us/query and 423.034 us/query
- Baseline parse/analyze/optimize: 1001.498 us/query
- Patched parse/analyze/optimize: 822.877 us/query and 844.032 us/query
The patch preserves optimizer behavior and only skips rules after a prior `Transformed::no` for the same unchanged plan version.
Contributor guide
Research direction
Start at the logical optimizer's rule-pass entry point and trace how each rule reports Transformed::no or transformed = true. Add plan-version tracking so unchanged no-op rules are skipped until a rule changes the plan, then verify fixed-point behavior and compare the ClickBench logical-planning measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100