OOM error with apache mina client, thousands of threads at jdk/internal/Unsafe.park(Native Method), java/lang/Object.waitImpl(Native Method)
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 400
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 1
Description
### Version
2.14.0
### Bug description
We are observing a large number of threads in Runnable state that never finish. These threads appear to be stuck in loops or waiting indefinitely, causing resource exhaustion over time. We are seeing excessive runnable threads in NioProcessor and SSHD Session timeout handling.
Environment:
OS: Windows Server 2022
JDK: 17.0.14+7
Apache MINA: 2.0.27
Apache MINA SSHD: 2.14.0
### Actual behavior
Threads remain in Runnable state for extended periods. Application performance degrades due to thread buildup. Threads remain active indefinitely, consuming CPU and memory and eventually triggering an OOM response.
### Expected behavior
Threads should terminate or block appropriately after completing their tasks.
### Relevant log output
```Shell
Nio:
at java/lang/Object.waitlmpl(Native Method)
at java/lang/Object.wait(Object.java:251(Compiled Code))
at java/lang/Object.wait(Object.java:219(Compiled Code))
at EDU/oswego/cs/dl/util/concurrent/Semaphore.attempt(Bytecode PC:62(Compiled Code))
at EDU/oswego/cs/dl/util/concurrent/SemaphoreControlledChannel.poll(Bytecode PC:19(Compiled Code))
at - (PsSelector.java:118(Compiled Code))
at - (PsSelector.java:208(Compiled Code))
at org/apache/mina/transport/socket/nio/NioProcessor.select(NioProcessor.java:112(Compiled Code))
at org/apache/mina/core/polling/AbstractPollingloProcessor$Processor.run(AbstractPollingloProcessor.java:616(Compiled Code)
at org/apache/mina/util/NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))
SSHD Client:
at java/util/concurrent/ConcurrentHashMap.remove(ConcurrentHashMap.java:1107(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.preProcessEncodeBuffer(AbstractSession.java:1069(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.encode(AbstractSession.java:1436(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.resolveOutputPacket(AbstractSession.java:1156(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.doWritePacket(AbstractSession.java:1164(Compiled Code))
at org/apache/sshd/common/session/helpers/KeyExchangeMessageHandler.writePacket(KeyExchangeMessageHandler.java:244(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.writePacket(AbstractSession.java:1087(Compiled Code))
at org/apache/sshd/common/session/Session.writePacket(Session.java:159(Compiled Code))
at org/apache/sshd/common/session/Session.writePacket(Session.java:145(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionHelper.disconnect(SessionHelper.java:1209(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionHelper.checkForTimeouts(SessionHelper.java:301(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionTimeoutListener.run(SessionTimeoutListener.java:84(Compiled Code))
at java/util/concurrent/Executors$RunnableAdapter.call(Executors.java:539(Compiled Code))
at java/util/concurrent/FutureTask.runAndReset(FutureTask.java:305(Compiled Code))
at java/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))
at jdk/internal/misc/Unsafe.park(Native Method)
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:252(Compiled Code))
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1679(Compiled Code)
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182(Compiled Code))
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))
```
### Other information
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the thread buildup with Apache MINA SSHD 2.14.0 on the stated JDK and inspect the NioProcessor and SessionTimeoutListener stack paths. Read NioProcessor.java, AbstractPollingIoProcessor, SessionTimeoutListener, and SessionHelper around the cited lines, then determine why threads remain active; done means the cause is explained and the workload no longer accumulates threads or reaches OOM.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100