opensearch-project / opensearch-project/sql
[FEATURE] Investigation on enable top-down optimizer in Calcite
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
Is your feature request related to a problem?
https://issues.apache.org/jira/browse/CALCITE-3916
Calcite has introduced top-down optimizer, which is more efficient than the old version. It will accelerate the optimizing process a lot by pruning. Let’s see if we should turn it on by default for our PPL optimization.
What solution would you like?
A clear and concise description of what you want to happen.
What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.
Do you have any additional context?
Enable top_down optimizer in Calcite will eliminate the issue https://github.com/opensearch-project/sql/issues/4842.It works becauseFilterMergeRuleisTransformationRulewhile our push down rules are not. So it happens to avoid intersection for this case. However it's a bigger problem if keeping push down rules non-transformation rule when using top down optimizer. See more explain in https://github.com/opensearch-project/sql/issues/4859#issuecomment-3588417949
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading Apache Calcite issue CALCITE-3916 and the discussion in issues #4842 and #4859 about the top-down optimizer and push-down rules. Determine whether enabling it by default is compatible with the project's rules, and document or implement a clear decision for PPL optimization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100