ServiceChannel gets stuck in failed Open state when attempting to auto close
- Dominant language
- C#
- Stars
- 1.8k
- Forks
- 576
- Avg merge
- 6d 9h
- Merged PRs (30d)
- 2
Description
**Describe the bug**
ServiceChannel stays in Open state after a graceful server shutdown. It has ```_hasChannelStartedAutoClosing``` set to true which ends up throwing ```ProtocolException``` on any usage attempts.
Even subsequent killing of the server process doesn't help getting it unstuck.
I'm using CoreWCF 1.7.0, NetNamedPipe transport, but being centered around NetFramingBase it could be reproduceable in other transports as well.
The sequence is as follows:
I call ```app.StopAsync()``` in my host. This begins shutdown sequence. However, this is not recognized properly by the client.
The sequence starts as expected:
```cs
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.TransportDuplexSessionChannel.CloseOutputSessionAsync(System.TimeSpan timeout) Line 267 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.TransportDuplexSessionChannel.ConnectionDuplexSession.CloseOutputSession(System.TimeSpan timeout) Line 641 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Channels.ServiceChannel.DecrementActivity() Line 843 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Channels.ServiceChannel.HandleReceiveComplete(System.ServiceModel.Channels.RequestContext context) Line 907 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Dispatcher.ChannelHandler.HandleReceiveComplete(System.ServiceModel.Channels.RequestContext context) Line 638 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Dispatcher.ChannelHandler.EndTryReceive(System.IAsyncResult result, out System.ServiceModel.Channels.RequestContext requestContext) Line 327 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult result) Line 191 C#
System.ServiceModel.Primitives.dll!System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(System.IAsyncResult result) Line 752 C#
```
At this point, ```_hasChannelStartedAutoClosing``` is set to true so all further calls to the service will end up with
```
System.ServiceModel.ProtocolException: 'This channel can no longer be used to send messages as the output session was auto-closed due to a server-initiated shutdown. Either disable auto-close by setting the DispatchRuntime.AutomaticInputSessionShutdown to false, or consider modifying the shutdown protocol with the remote server.'
```
Next, ```CloseOutputSessionCoreAsync()``` is called, which succeeds.
But the following sequence is what baffles me: the ServiceChannel stays Open, there's no event notification to Closing or Closed events, there's no way for my code to check if it's alive (aside from receiving ```System.ServiceModel.ProtocolException``` on any call) while the connection is being pushed into the connection pool.
```cs
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.CommunicationPool.EndpointConnectionPool.AddConnection(System.ServiceModel.Channels.IConnection connection, System.TimeSpan timeout) Line 407 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.CommunicationPool.AddConnection(string key, System.ServiceModel.Channels.IConnection connection, System.TimeSpan timeout) Line 198 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.ConnectionPoolHelper.ReleaseConnection(bool abort, System.TimeSpan timeout) Line 206 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.ConnectionPoolHelper.Close(System.TimeSpan timeout) Line 167 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.ReturnConnectionIfNecessary(bool abort, System.TimeSpan timeout) Line 314 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.TransportDuplexSessionChannel.OnOutputSessionClosed(ref System.Runtime.TimeoutHelper timeoutHelper) Line 570 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.TransportDuplexSessionChannel.CloseOutputSessionAsync(System.TimeSpan timeout) Line 271 C#
```
As an attempted fix, I've set on my Binding ```MaxConnections = 0``` which forces ```IdleConnections.Add()``` to return false.
This leads to even more baffling situation:
```cs System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.ConnectionPool.CloseItemAsync(System.ServiceModel.Channels.IConnection item, System.TimeSpan timeout) Line 640 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.CommunicationPool.EndpointConnectionPool.CloseItemAsync(System.ServiceModel.Channels.IConnection item, System.TimeSpan timeout) Line 290 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.IdlingCommunicationPool.IdleTimeoutEndpointConnectionPool.CloseItemAsync(System.ServiceModel.Channels.IConnection item, System.TimeSpan timeout) Line 69 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.CommunicationPool.EndpointConnectionPool.CloseIdleConnection(System.ServiceModel.Channels.IConnection connection, System.TimeSpan timeout) Line 521 C#
System.ServiceModel.NetFramingBase.dll!System.ServiceModel.Channels.CommunicationPool.EndpointConnectionPool.AddConnection(System.ServiceModel.Channels.IConnection connection, System.TimeSpan timeout) Line 409 C#
```
It closes my connection... and then nothing. ServiceChannel still stays in a glitched Open state.
So, I would assume that ```OnOutputSessionClosed``` should transition the ServiceChannel into a Closed state first and return the connection to the pool later.
**To Reproduce**
Create a minimal NetNamedPipe based CoreWCF server
Call ```app.StopAsync()```
Observe channel state on the client
**Expected behavior**
Channel sends Closing, then Closed event and changes its state to Closed so that client code could react to a server shutdown accordingly
Also reported for NetTcp connections here:
CoreWCF/CoreWCF#1402
Contributor guide
Assessment
This issue has not been assessed yet.