opensearch-project / opensearch-project/sql

[FEATURE] Let `sql-worker` thread pool be resizeable instead of fixed

Open
#5,357 3 comments 0 reactions 1 assignee View on GitHub

@jhansimandava27-lang is already working on this.

Since Apr 29, 2026.

enhancement performance PPL SQL
Dominant language
Java
Stars
176
Forks
229
Avg merge
2d 21h
Merged PRs (30d)
43

Description

Is your feature request related to a problem?
When running DSL vs PPL load tests on a managed OpenSearch 3.5 domain, I found that DSL's bottleneck was the entire cluster, while PPL's bottleneck was the sql-worker thread pool.

DSL queries execute on the resizeable search thread pool. When running expensive DSL queries on increasingly high volume data, it dynamically scales up to use more of the cluster's resources in response to the increased load. DSL performance only degraded when the cluster itself was strained from the load.

PPL queries execute on the fixed sql-worker thread pool. When running expensive PPL queries on increasingly high volume data, this thread pool eventually saturated. As a result, PPL tasks waited long times in thread pool queues, and PPL specifically degraded while the rest of the cluster moved on without issue.

What solution would you like?
Change the construction here to use a dynamic executor builder instead of a fixed executor builder: https://github.com/opensearch-project/sql/blob/ee28e3d106d30cd1e70fed9a35444da40683b7ec/plugin/src/main/java/org/opensearch/sql/plugin/SQLPlugin.java#L354-L360

What alternatives have you considered?
Deliberation may be needed on if this is a desirable change, and consideration of potential consequences for PPL work competing for cluster resources with other tasks.

Do you have any additional context?
N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.