alibaba / alibaba/DataX

[Bug] Critical OS Thread Leak (OOM) via unmanaged ThreadLocal<ExecutorService> in DBUtil.java

Open
#2,346 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
A severe "nested concurrency" leak exists in `com.alibaba.datax.plugin.rdbms.util.DBUtil`. The class defines a `ThreadLocal` named `rsExecutors` that initializes a new `FixedThreadPool(1)` for every calling thread. However, there is no mechanism to clean this up: `ExecutorService.shutdown()` and `rsExecutors.remove()` are never invoked.

**Impact**
Because DataX is a highly concurrent data synchronization framework, multiple task threads will access `DBUtil`. Each of these worker threads will secretly spawn its own dedicated underlying thread pool that lives forever.
This causes an exponential explosion of underlying OS threads. Eventually, this exhausts the operating system's `ulimit` thread limit, directly crashing the JVM with a `java.lang.OutOfMemoryError: unable to create new native thread`.

**Suggested Fix**
The use of `ThreadLocal` to cache thread pools is an anti-pattern here.
Remove the `ThreadLocal` wrapper entirely. Declare a single, globally shared `static ExecutorService` (e.g., a cached thread pool with a proper timeout and rejection policy) for all `asyncResultSetNext` calls to reuse. This will immediately resolve the thread explosion issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in DBUtil.java at the rsExecutors ThreadLocal and trace its use from asyncResultSetNext. Confirm how each calling thread creates and retains an ExecutorService, then verify the change removes the per-thread leak while preserving asynchronous result-set behavior and preventing unbounded native-thread growth.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.