DgraphAsyncClient blocks ForkJoinPool.commonPool threads and can starve the JVM common pool

DgraphAsyncClient runs its async work on ForkJoinPool.commonPool() by default and blocks those threads for the full duration of each gRPC call. Under sustained or slow traffic this can exhaust the JVM-wide common pool and hang anything else in the process that depends on it (parallel streams, other CompletableFuture chains).

Originally reported previously:

Hi, we have noticed that dgraph4j DgraphAsyncClient is not closing connections properly.

It uses ForkJoinPool.commonPool which is JVM's fork join pool. Eventually, it would fill up and hang all execution.

Please investigate/fix, we are attempting to work around this for the moment by giving it a separate forkjoin pool and adding more keep alives and timeouts to requests but that's downstream of the client.

Root cause

Two things compound here:

1. The default constructor sets this.executor = ForkJoinPool.commonPool() (DgraphAsyncClient.java#L54).
2. CompletableFutures.runWithRetries wraps every operation in CompletableFuture.supplyAsync(...) on that executor and then calls a blocking .get() on the inner gRPC future (CompletableFutures.java#L46). The JWT-expiry retry path blocks the same thread twice more (retryLogin.get().get(), then a second ctxCallable.call().get()).

So every alter, query, mutation, checkVersion, runDQL, etc. parks a common-pool thread until the gRPC round trip completes. The blocking call is not wrapped in a ForkJoinPool.ManagedBlocker, so the pool never compensates with extra threads. Since commonPool parallelism is roughly CPU cores minus one and shared across the whole JVM, a modest burst of slow or hung requests (server under load, network stall, missing deadlines) starves the pool. The reporter's workaround (a dedicated executor via the DgraphAsyncClient(Executor, DgraphStub...) constructor) is consistent with this diagnosis.

Note the "futures are not properly closed" framing in the original report is likely a misread of the symptom; the futures complete fine, the threads they run on are just blocked.

Suggested fix

runWithRetries doesn't actually need to block at all. The stub already returns a CompletableFuture (via StreamObserverBridge), so the method can compose instead of parking a thread: thenCompose on the stub future and handle the JWT-expiry retry with an exceptional-completion stage. That removes the blocking .get() calls entirely and makes the executor a callback executor rather than a thread the request holds hostage.

Shorter-term mitigations, if a full rewrite isn't wanted:

• Default to a small dedicated executor instead of ForkJoinPool.commonPool().
• Wrap the blocking section in a ForkJoinPool.ManagedBlocker so the common pool can compensate.
• At minimum, document that the default constructor blocks common-pool threads and that production users should supply their own executor.

Affected versions

Present on main as of v25.0.0; the same pattern exists in earlier releases (previously inline in DgraphAsyncClient.runWithRetries before the extraction to CompletableFutures).

Start with src/main/java/io/dgraph/DgraphAsyncClient.java and src/main/java/io/dgraph/CompletableFutures.java, especially the default executor, runWithRetries, and JWT retry path. Trace the gRPC future handling and verify that async operations and retries no longer block ForkJoinPool.commonPool threads while preserving their existing behavior.