Core: Static thread pools in ThreadPools.java cause ClassLoader leaks in hot-reload scenarios
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
main (development)
### Query engine
None
### Please describe the bug 🐞
### Context
The `org.apache.iceberg.util.ThreadPools` class maintains static `ExecutorService` instances (`WORKER_POOL` and `DELETE_WORKER_POOL`) that are initialized statically and persist for the lifetime of the JVM.
### Problem
These thread pools are created using `MoreExecutors.getExitingExecutorService`, which registers a **JVM shutdown hook** to terminate the threads. However, there is no public API to explicitly shut down these pools when the Iceberg library is unloaded or when the application using Iceberg is stopped (without killing the JVM).
### Impact
In environments that support hot-reloading or dynamic lifecycles (e.g., Tomcat Web Containers, Flink clusters, OSGi), the JVM does not exit when an application is undeployed.
1. The static thread pools remain active.
2. The threads inside these pools retain references to their `inheritedAccessControlContext` or `contextClassLoader`.
3. This prevents the application's `ClassLoader` from being garbage collected.
4. Repeated redeployments lead to a **ClassLoader Leak** and eventually `java.lang.OutOfMemoryError: Metaspace`.
### Analysis from Source Code
In `ThreadPools.java`:
```java
private static final ExecutorService WORKER_POOL =
newExitingWorkerPool("iceberg-worker-pool", WORKER_THREAD_POOL_SIZE);
The WORKER_POOL is private static final and depends solely on SystemConfigs. There is no mechanism to release these resources programmatically.
### Willingness to contribute
- [] I can contribute a fix for this bug independently
- [] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
Contributor guide
Assessment
This issue has not been assessed yet.