[BUG] When Admin is in cluster mode, the restart causes the gateway node to keep reconnecting by websocket
- Dominant language
- Java
- Stars
- 8.8k
- Forks
- 3.1k
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 85
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Current Behavior
1. admin和网关都是k8s部署
2. admin配置集群模式
3. 网关使用k8s本地域名进行ws连接
4. 使用websocket同步数据
5. 重启admin
网关日志:2026-07-29 14:29:31.078 INFO [] ,,,WebSocketConnectReadThread-121,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending sync msg
2026-07-29 14:29:31.078 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is successful.....
2026-07-29 14:30:31.079 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,shenyu bootstrap websocket namespaceId: 403dfe7c-e94c-4582-8e71-994e8356f961
2026-07-29 14:30:31.082 INFO [] ,,,WebSocketConnectReadThread-124,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending sync msg
2026-07-29 14:30:31.082 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is successful.....
2026-07-29 14:31:31.082 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,shenyu bootstrap websocket namespaceId: 403dfe7c-e94c-4582-8e71-994e8356f961
2026-07-29 14:31:31.085 INFO [] ,,,WebSocketConnectReadThread-127,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending sync msg
2026-07-29 14:31:31.085 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is successful.....
2026-07-29 14:32:31.085 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,shenyu bootstrap websocket namespaceId: 403dfe7c-e94c-4582-8e71-994e8356f961
2026-07-29 14:32:31.089 INFO [] ,,,WebSocketConnectReadThread-130,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending sync msg
2026-07-29 14:32:31.089 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is successful.....
2026-07-29 14:33:31.089 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,shenyu bootstrap websocket namespaceId: 403dfe7c-e94c-4582-8e71-994e8356f961
2026-07-29 14:33:31.092 INFO [] ,,,WebSocketConnectReadThread-133,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending sync msg
2026-07-29 14:33:31.093 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is successful.....
2026-07-29 14:34:31.094 INFO [] ,,,shenyu-shared_wheel_timer-4,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,shenyu bootstrap websocket namespaceId: 403dfe7c-e94c-4582-8e71-994e8356f961
2026-07-29 14:34:31.097 INFO [] ,,,WebSocketConnectReadThread-136,o.a.s.p.s.d.w.c.ShenyuWebsocketClient,websocket connection server[ws://shenyu-admin.base.svc.cluster.local/websocket] is opened, sending syn
Admin日志
2026-07-29 15:08:31.249 ERROR [] ,,,http-nio-0.0.0.0-80-exec-10,o.a.s.a.l.w.WebsocketCollector,websocket send result is exception: java.io.IOException: java.io.IOException: Broken pipe
1785308911249 2026-07-29T07:08:31.249Z 2026-07-29 15:08:31.249 WARN [] ,,,http-nio-0.0.0.0-80-exec-10,o.a.s.a.l.w.WebsocketCollector,websocket close on client[10.10.16.173]
1785308911249 2026-07-29T07:08:31.249Z
1785308911249 2026-07-29T07:08:31.249Z ... 37 common frames omitted
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlockInternal(WsRemoteEndpointImplBase.java:313)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:521)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:171)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:1337)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:1366)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.SocketWrapperBase.vectoredOperation(SocketWrapperBase.java:1440)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.SocketWrapperBase$OperationState.start(SocketWrapperBase.java:1051)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper$NioOperationState.run(NioEndpoint.java:1632)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:134)
1785308911249 2026-07-29T07:08:31.249Z at java.base/sun.nio.ch.SocketChannelImpl.write(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at java.base/sun.nio.ch.IOUtil.write(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at java.base/sun.nio.ch.IOUtil.write(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at java.base/sun.nio.ch.SocketDispatcher.writev(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at java.base/sun.nio.ch.SocketDispatcher.writev0(Native Method)
1785308911249 2026-07-29T07:08:31.249Z Caused by: java.io.IOException: Broken pipe
1785308911249 2026-07-29T07:08:31.249Z at java.base/java.lang.Thread.run(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:904)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:57)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:60)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler.upgradeDispatch(WsHttpUpgradeHandler.java:152)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsFrameServer.notifyDataAvailable(WsFrameServer.java:164)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsFrameServer.doOnDataAvailable(WsFrameServer.java:184)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:85)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsFrameBase.processInputBuffer(WsFrameBase.java:130)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsFrameBase.processData(WsFrameBase.java:284)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsFrameBase.processDataText(WsFrameBase.java:484)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.server.WsFrameServer.sendMessageText(WsFrameServer.java:130)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsFrameBase.sendMessageText(WsFrameBase.java:390)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.pojo.PojoMessageHandlerWholeBase.onMessage(PojoMessageHandlerWholeBase.java:102)
1785308911249 2026-07-29T07:08:31.249Z at java.base/java.lang.reflect.Method.invoke(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.websocket.WebsocketCollector.onMessage(WebsocketCollector.java:203)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.service.impl.SyncDataServiceImpl.syncAllByNamespaceId(SyncDataServiceImpl.java:153)
1785308911249 2026-07-29T07:08:31.249Z at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384)
1785308911249 2026-07-29T07:08:31.249Z at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451)
1785308911249 2026-07-29T07:08:31.249Z at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156)
1785308911249 2026-07-29T07:08:31.249Z at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178)
1785308911249 2026-07-29T07:08:31.249Z at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.DataChangedEventDispatcher.onApplicationEvent(DataChangedEventDispatcher.java:52)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.DataChangedEventDispatcher.onApplicationEvent(DataChangedEventDispatcher.java:99)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.websocket.WebsocketDataChangedListener.onSelectorChanged(WebsocketDataChangedListener.java:84)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.websocket.WebsocketCollector.send(WebsocketCollector.java:279)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.shenyu.admin.listener.websocket.WebsocketCollector.sendMessageBySession(WebsocketCollector.java:293)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointBasic.sendText(WsRemoteEndpointBasic.java:36)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendString(WsRemoteEndpointImplBase.java:192)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:250)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:266)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlockInternal(WsRemoteEndpointImplBase.java:324)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:589)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsSession.sendCloseMessage(WsSession.java:789)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:266)
1785308911249 2026-07-29T07:08:31.249Z at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlockInternal(WsRemoteEndpointImplBase.java:326)
1785308911249 2026-07-29T07:08:31.249Z 2026-07-29 15:08:31.249 ERROR [] ,,,http-nio-0.0.0.0-80-exec-10,o.a.s.a.l.w.WebsocketCollector,websocket collection on client[10.10.16.173] error: java.io.IOException: java.io.IOException: Broken pipe
每次Admin准备写回数据,ws连接就关闭了
### Expected Behavior
正常连接到master
### Steps To Reproduce
1. admin和网关都是k8s部署
2. admin配置集群模式
3. 网关使用k8s本地域名进行ws连接
4. 使用websocket同步数据
5. 重启admin
### Environment
```markdown
ShenYu version(s): 2.7.0.3
```
### Debug logs
_No response_
### Anything else?
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the WebsocketCollector, WebsocketDataChangedListener, and SyncDataServiceImpl locations shown in the stack trace, then reproduce the Kubernetes cluster-mode restart sequence while watching gateway and admin logs. Trace how the gateway reconnects and how the admin selects the master session after restart. Done means the gateway remains connected to the master and no longer repeatedly reconnects or produces broken-pipe errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kubernetes
- Domain
- backend, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100