openai / openai/openai-java

Allow custom executor/thread pool for async HTTP calls in CompletionServiceAsyncImpl

Open
#754 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Kotlin
Stars
1.5k
Forks
264
Avg merge
9h 46m
Merged PRs (30d)
96

Description

Description

CompletionServiceAsyncImpl currently uses the JVM ForkJoinPool.commonPool() to invoke async HTTP calls:

https://github.com/openai/openai-java/blob/main/openai-java-core/src/main/kotlin/com/openai/services/async/CompletionServiceAsyncImpl.kt#L143

In enterprise environments, relying on the global common pool can break tracing, observability, and context propagation flows. Many applications use custom executors/thread pools to preserve tracing context, MDC/logging context, OpenTelemetry context, or other request-scoped metadata across async boundaries.

Because the common pool is not customizable, users cannot currently integrate the OpenAI Java client cleanly with their existing execution and observability infrastructure.

Problem

There is currently no supported way to override the thread pool used by CompletionServiceAsyncImpl.

The only workaround is to copy the generated/service implementation class and modify the executor usage manually, which is fragile and difficult to maintain across library upgrades.

Proposed Solution

Provide an option to configure the executor/thread pool when building the client.

For example:

val client = OpenAIClient.builder()
.apiKey(apiKey)
.executor(customExecutor)
.build()

or, if scoped specifically to async execution:

val client = OpenAIClient.builder()
.apiKey(apiKey)
.asyncExecutor(customExecutor)
.build()

The async service implementation could then use the configured executor instead of ForkJoinPool.commonPool().

Expected Behavior

Users should be able to supply a custom executor so async HTTP calls run on infrastructure-controlled threads, allowing tracing, observability, and context propagation to work correctly.

Current Workaround

The current workaround is to copy CompletionServiceAsyncImpl and modify the implementation to use a custom executor, but this is not ideal because it forks library internals and creates maintenance risk.

Additional Context

This is especially important in enterprise scenarios where applications require strict control over execution context, thread naming, tracing propagation, and monitoring behavior.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openai-java-core/src/main/kotlin/com/openai/services/async/CompletionServiceAsyncImpl.kt at the referenced executor usage, then trace OpenAIClient.builder() to find where client configuration is assembled. Determine how the configured executor should reach the async service, and verify that async HTTP calls use it instead of ForkJoinPool.commonPool().

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.