opensearch-project / opensearch-project/sql
Process spark query once in sql repo for handling validations and other operations.
Open
Nobody has claimed this yet.
catch-all acknowledged
enhancement
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
More details, comments in this PR: https://github.com/opensearch-project/sql/pull/2884/files
Today, we might be running spark query through parser multiple times, ideally we should be parsing it once.
Analyze and optimize these operations.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the additional details and comments in PR 2884, especially how Spark queries pass through parsing for validation and other operations. Trace the current repeated parsing flow and compare the operations before and after optimization; done means the query is parsed once without changing validation or other behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark, sql
- Domain
- backend, data-engineering, databases
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100