eclipse-ee4j / eclipse-ee4j/tyrus

Tyrus WebSocket sporadic client side disconnects

Open
#722 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
128
Forks
49
PR merge metrics
No merged PRs in 30d

Description

I'm using Tyrus client to connect to multiple WebSocket servers (around 10 connections) in parallel. The servers publish a lot of messages (hundreds of millions of messages per day). At some point, one of the connections begins to experience client side disconnects. The `onClose` handlers is invoked with close reason `CloseReason[1006,Closed abnormally.]`. The `onError` handler is not invoked. After the error, the connection is re-established using the `ReconnectHandler`.

Couple of facts regarding those disconnects/reconnects:

- It's always the connection to the same server that has issues. All other connections behave as expected.
- When I estalish the connection to just that one server that is having issues then no disconnects take place
- The first disconnect occurs after about an hour after the start of the application
- Then disconnects happen every few minutes

I also have the stack trace from the onError handler:

```
2020-11-16 16:23:03,347 3985311 [Grizzly(2)] INFO MyEndpoint - java.lang.Throwable
at MyEndpoint.onClose(MyEndpoint.scala:185)
at org.glassfish.tyrus.core.TyrusEndpointWrapper.onClose(TyrusEndpointWrapper.java:1235)
at org.glassfish.tyrus.core.TyrusWebSocket.onClose(TyrusWebSocket.java:110)
at org.glassfish.tyrus.core.ProtocolHandler.close(ProtocolHandler.java:481)
at org.glassfish.tyrus.core.TyrusWebSocket.close(TyrusWebSocket.java:244)
at org.glassfish.tyrus.client.TyrusClientEngine$2$1.close(TyrusClientEngine.java:613)
at org.glassfish.tyrus.container.grizzly.client.GrizzlyClientFilter$CloseTask.execute(GrizzlyClientFilter.java:470)
at org.glassfish.tyrus.container.grizzly.client.TaskProcessor.processTask(TaskProcessor.java:91)
at org.glassfish.tyrus.container.grizzly.client.TaskProcessor.processTask(TaskProcessor.java:68)
at org.glassfish.tyrus.container.grizzly.client.GrizzlyClientFilter.handleClose(GrizzlyClientFilter.java:197)
at org.glassfish.grizzly.filterchain.ExecutorResolver$4.execute(ExecutorResolver.java:50)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:248)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:181)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:121)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:99)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
at org.glassfish.grizzly.nio.NIOConnection.preClose(NIOConnection.java:819)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.preClose(TCPNIOConnection.java:78)
at org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:557)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:249)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.read(TCPNIOTransport.java:596)
at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleRead(TCPNIOTransportFilter.java:59)
at org.glassfish.grizzly.filterchain.TransportFilter.handleRead(TransportFilter.java:133)
at org.glassfish.grizzly.ssl.SSLBaseFilter$SSLTransportFilterWrapper.handleRead(SSLBaseFilter.java:919)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:248)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:181)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:121)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:99)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:34)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
at java.base/java.lang.Thread.run(Thread.java:834)
```

Here is (roughly) the code I'm using

``` scala
import jakarta.websocket._

class MyEndpoint extends Endpoint with LazyLogging {
def onOpen(session: Session, config: EndpointConfig): Unit = {
session.addMessageHandler(new MessageHandler.Whole[String] {
def onMessage(message: String): Unit = {
// handle message
}

override def onError(session: Session, thr: Throwable): Unit = {
logger.error(s"error occurred", thr)

super.onError(session, thr)
}

override def onClose(session: Session, closeReason: CloseReason): Unit = {
logger.error(s"closing due to $closeReason")

logger.info(ExceptionUtils.getStackTrace(new Throwable()))

super.onClose(session, closeReason)
}

}
}
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.