eclipse-ee4j / eclipse-ee4j/tyrus
Tyrus WebSocket sporadic client side disconnects
- Dominant language
- Java
- Stars
- 128
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
I'm using Tyrus client to connect to multiple WebSocket servers (around 10 connections) in parallel. The servers publish a lot of messages (hundreds of millions of messages per day). At some point, one of the connections begins to experience client side disconnects. The `onClose` handlers is invoked with close reason `CloseReason[1006,Closed abnormally.]`. The `onError` handler is not invoked. After the error, the connection is re-established using the `ReconnectHandler`.
Couple of facts regarding those disconnects/reconnects:
- It's always the connection to the same server that has issues. All other connections behave as expected.
- When I estalish the connection to just that one server that is having issues then no disconnects take place
- The first disconnect occurs after about an hour after the start of the application
- Then disconnects happen every few minutes
I also have the stack trace from the onError handler:
```
2020-11-16 16:23:03,347 3985311 [Grizzly(2)] INFO MyEndpoint - java.lang.Throwable
at MyEndpoint.onClose(MyEndpoint.scala:185)
at org.glassfish.tyrus.core.TyrusEndpointWrapper.onClose(TyrusEndpointWrapper.java:1235)
at org.glassfish.tyrus.core.TyrusWebSocket.onClose(TyrusWebSocket.java:110)
at org.glassfish.tyrus.core.ProtocolHandler.close(ProtocolHandler.java:481)
at org.glassfish.tyrus.core.TyrusWebSocket.close(TyrusWebSocket.java:244)
at org.glassfish.tyrus.client.TyrusClientEngine$2$1.close(TyrusClientEngine.java:613)
at org.glassfish.tyrus.container.grizzly.client.GrizzlyClientFilter$CloseTask.execute(GrizzlyClientFilter.java:470)
at org.glassfish.tyrus.container.grizzly.client.TaskProcessor.processTask(TaskProcessor.java:91)
at org.glassfish.tyrus.container.grizzly.client.TaskProcessor.processTask(TaskProcessor.java:68)
at org.glassfish.tyrus.container.grizzly.client.GrizzlyClientFilter.handleClose(GrizzlyClientFilter.java:197)
at org.glassfish.grizzly.filterchain.ExecutorResolver$4.execute(ExecutorResolver.java:50)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:248)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:181)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:121)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:99)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
at org.glassfish.grizzly.nio.NIOConnection.preClose(NIOConnection.java:819)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.preClose(TCPNIOConnection.java:78)
at org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:557)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:249)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.read(TCPNIOTransport.java:596)
at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleRead(TCPNIOTransportFilter.java:59)
at org.glassfish.grizzly.filterchain.TransportFilter.handleRead(TransportFilter.java:133)
at org.glassfish.grizzly.ssl.SSLBaseFilter$SSLTransportFilterWrapper.handleRead(SSLBaseFilter.java:919)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:248)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:181)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:121)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:99)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:34)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
at java.base/java.lang.Thread.run(Thread.java:834)
```
Here is (roughly) the code I'm using
``` scala
import jakarta.websocket._
class MyEndpoint extends Endpoint with LazyLogging {
def onOpen(session: Session, config: EndpointConfig): Unit = {
session.addMessageHandler(new MessageHandler.Whole[String] {
def onMessage(message: String): Unit = {
// handle message
}
override def onError(session: Session, thr: Throwable): Unit = {
logger.error(s"error occurred", thr)
super.onError(session, thr)
}
override def onClose(session: Session, closeReason: CloseReason): Unit = {
logger.error(s"closing due to $closeReason")
logger.info(ExceptionUtils.getStackTrace(new Throwable()))
super.onClose(session, closeReason)
}
}
}
}
```
Contributor guide
Assessment
This issue has not been assessed yet.