apache / apache/mina-sshd

OOM error with apache mina client, thousands of threads at jdk/internal/Unsafe.park(Native Method), java/lang/Object.waitImpl(Native Method)

Open
#859 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.1k
Forks
400
Avg merge
5d 1h
Merged PRs (30d)
1

Description

### Version

2.14.0

### Bug description

We are observing a large number of threads in Runnable state that never finish. These threads appear to be stuck in loops or waiting indefinitely, causing resource exhaustion over time. We are seeing excessive runnable threads in NioProcessor and SSHD Session timeout handling.

Environment:
OS: Windows Server 2022
JDK: 17.0.14+7
Apache MINA: 2.0.27
Apache MINA SSHD: 2.14.0

### Actual behavior

Threads remain in Runnable state for extended periods. Application performance degrades due to thread buildup. Threads remain active indefinitely, consuming CPU and memory and eventually triggering an OOM response.

### Expected behavior

Threads should terminate or block appropriately after completing their tasks.

### Relevant log output

```Shell
Nio:

at java/lang/Object.waitlmpl(Native Method)
at java/lang/Object.wait(Object.java:251(Compiled Code))
at java/lang/Object.wait(Object.java:219(Compiled Code))
at EDU/oswego/cs/dl/util/concurrent/Semaphore.attempt(Bytecode PC:62(Compiled Code))
at EDU/oswego/cs/dl/util/concurrent/SemaphoreControlledChannel.poll(Bytecode PC:19(Compiled Code))
at - (PsSelector.java:118(Compiled Code))
at - (PsSelector.java:208(Compiled Code))
at org/apache/mina/transport/socket/nio/NioProcessor.select(NioProcessor.java:112(Compiled Code))
at org/apache/mina/core/polling/AbstractPollingloProcessor$Processor.run(AbstractPollingloProcessor.java:616(Compiled Code)
at org/apache/mina/util/NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))

SSHD Client:
at java/util/concurrent/ConcurrentHashMap.remove(ConcurrentHashMap.java:1107(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.preProcessEncodeBuffer(AbstractSession.java:1069(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.encode(AbstractSession.java:1436(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.resolveOutputPacket(AbstractSession.java:1156(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.doWritePacket(AbstractSession.java:1164(Compiled Code))
at org/apache/sshd/common/session/helpers/KeyExchangeMessageHandler.writePacket(KeyExchangeMessageHandler.java:244(Compiled Code))
at org/apache/sshd/common/session/helpers/AbstractSession.writePacket(AbstractSession.java:1087(Compiled Code))
at org/apache/sshd/common/session/Session.writePacket(Session.java:159(Compiled Code))
at org/apache/sshd/common/session/Session.writePacket(Session.java:145(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionHelper.disconnect(SessionHelper.java:1209(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionHelper.checkForTimeouts(SessionHelper.java:301(Compiled Code))
at org/apache/sshd/common/session/helpers/SessionTimeoutListener.run(SessionTimeoutListener.java:84(Compiled Code))
at java/util/concurrent/Executors$RunnableAdapter.call(Executors.java:539(Compiled Code))
at java/util/concurrent/FutureTask.runAndReset(FutureTask.java:305(Compiled Code))
at java/util/concurrent/ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))

at jdk/internal/misc/Unsafe.park(Native Method)
at java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.java:252(Compiled Code))
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1679(Compiled Code)
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182(Compiled Code))
at java/util/concurrent/ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122(Compiled Code))
at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635(Compiled Code))
at java/lang/Thread.run(Thread.java:853(Compiled Code))
```

### Other information

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the thread buildup with Apache MINA SSHD 2.14.0 on the stated JDK and inspect the NioProcessor and SessionTimeoutListener stack paths. Read NioProcessor.java, AbstractPollingIoProcessor, SessionTimeoutListener, and SessionHelper around the cited lines, then determine why threads remain active; done means the cause is explained and the workload no longer accumulates threads or reaches OOM.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.