opensearch-project / opensearch-project/sql
[Enhancement] Untangle opensearch dependencies from async api query and create a core module.
Open
@vamsimanohar is already working on this.
Since Apr 17, 2024.
enhancement
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
Motivation:
The current state of async query APIs in the Spark module has several limitations and challenges that need to be addressed. The async query APIs are tightly coupled with OpenSearch dependencies, which are spread across the codebase. This tight coupling makes it difficult to adopt and utilize the async query functionality in cloud-native applications.
New Module Structure
- async-query-core: This module contains the core business logic without concrete implementations of storage and configs. It should not have any dependencies on OpenSearch functionality.
- async-query-rest: This module contains the REST layer along with the necessary implementations of storage and config. It can still exist in the Spark module or can be separated out.
Plan to Achieve
- Refactor all the cluster settings dependencies:
* Create config supplier interfaces to abstract the cluster settings.
* Implement concrete implementations of the config supplier interfaces using OpenSearch cluster settings. - Refactor all the storage dependencies:
* Define appropriate storage service interfaces to abstract the storage functionality.
* Implement default storage service implementations using OpenSearch as the underlying storage. - Refactor all the NodeClient dependencies:
* Create appropriate interfaces to abstract out the NodeClient functionality.
* Implement the necessary interfaces to replace direct usage of NodeClient. - Move the core logic and new interfaces to the new async-query-core module:
* Identify and extract the core business logic that does not depend on OpenSearch functionality.
* Move the extracted core logic and newly created interfaces to the async-query-core module.
* Ensure that the async-query-core module does not have any dependencies on OpenSearch.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.