Is it possible to identify whether an IO operation failed due to the connector being forcefully stopped?

Jetty version(s)

Jetty 12.0.22

Enhancement Description

When I am shutting down a Jetty server I want to be able to quickly shutdown by terminating any ongoing requests, this currently seems to be possible by calling Connector::stop.

However I want to be able to continue emitting logs and other telemetry data, so that I can see the reason that any given request failed.

However once the connector has stopped, the IOExceptions are ambiguous, it isn't clear if the exception happened because the client disconnected, or because the server disconnected due to shutdown. As a result this makes it more challenging to ensure accurate logging.

Would it be feasible to have a more useful IOException class/message saying that the connection was stopped?

Example exceptions

Write response failure due to shutdown:

java.io.EOFException: reset
	at org.eclipse.jetty.http2.HTTP2Stream.reset(HTTP2Stream.java:182)
	at org.eclipse.jetty.http2.HTTP2Session.failStreams(HTTP2Session.java:724)
	at org.eclipse.jetty.http2.HTTP2Session.failStreams(HTTP2Session.java:704)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.halt(HTTP2Session.java:1965)
	at org.eclipse.jetty.http2.HTTP2Session.doStop(HTTP2Session.java:175)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
	at org.eclipse.jetty.util.component.LifeCycle.stop(LifeCycle.java:91)
	at org.eclipse.jetty.http2.SessionContainer.onClosed(SessionContainer.java:79)
	at org.eclipse.jetty.io.AbstractConnection.onClosed(AbstractConnection.java:247)
	at org.eclipse.jetty.io.AbstractConnection.onClose(AbstractConnection.java:239)
	at org.eclipse.jetty.http2.HTTP2Connection.onClose(HTTP2Connection.java:147)
	at org.eclipse.jetty.io.ssl.SslConnection.onClose(SslConnection.java:358)
	at org.eclipse.jetty.io.SelectorManager.connectionClosed(SelectorManager.java:347)
	at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:1094)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Read request failure due to shutdown:

org.eclipse.jetty.io.EofException: null
	at org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2.onFailure(HttpStreamOverHTTP2.java:594)
	at org.eclipse.jetty.http2.server.internal.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:225)
	at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:180)
	at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:175)
	at org.eclipse.jetty.http2.HTTP2Stream.notifyFailure(HTTP2Stream.java:937)
	at org.eclipse.jetty.http2.HTTP2Stream.onFailure(HTTP2Stream.java:630)
	at org.eclipse.jetty.http2.HTTP2Stream.process(HTTP2Stream.java:376)
	at org.eclipse.jetty.http2.HTTP2Session.failStream(HTTP2Session.java:730)
	at org.eclipse.jetty.http2.HTTP2Session.failStreams(HTTP2Session.java:722)
	at org.eclipse.jetty.http2.HTTP2Session.failStreams(HTTP2Session.java:704)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.halt(HTTP2Session.java:1965)
	at org.eclipse.jetty.http2.HTTP2Session.doStop(HTTP2Session.java:175)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
	at org.eclipse.jetty.util.component.LifeCycle.stop(LifeCycle.java:91)
	at org.eclipse.jetty.http2.SessionContainer.onClosed(SessionContainer.java:79)
	at org.eclipse.jetty.io.AbstractConnection.onClosed(AbstractConnection.java:247)
	at org.eclipse.jetty.io.AbstractConnection.onClose(AbstractConnection.java:239)
	at org.eclipse.jetty.http2.HTTP2Connection.onClose(HTTP2Connection.java:147)
	at org.eclipse.jetty.io.ssl.SslConnection.onClose(SslConnection.java:358)
	at org.eclipse.jetty.io.SelectorManager.connectionClosed(SelectorManager.java:347)
	at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:1094)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.eclipse.jetty.io.EofException: null
	at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:174)
	... 21 common frames omitted
Caused by: java.io.IOException: cancel_stream_error/stop
	at org.eclipse.jetty.http2.HTTP2Session.toFailure(HTTP2Session.java:735)
	at org.eclipse.jetty.http2.HTTP2Session.failStreams(HTTP2Session.java:703)
	... 15 common frames omitted

Work around

I think as a work-around I can check every time that I get an exception from a ServletInputStream / ServletOutputStream to see if the connector has been closed. However this will be fiddly and not guaranteed to be 100% reliable in case of race conditions.

Start by tracing Connector::stop through HTTP2Session.doStop, failStreams, HTTP2Stream.reset and HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure. Compare the shutdown stack traces with client-disconnect handling in the listed HTTP/2 and I/O classes. Done means callers can reliably distinguish connector shutdown failures from client disconnects when handling ServletInputStream or ServletOutputStream exceptions.