Idle timeout on already completed channel throws IllegalStateException("channel already completed") from HttpChannelState$ChannelResponse.write()
- Dominant language
- Java
- Stars
- 4.1k
- Forks
- 2k
- Avg merge
- 3d 56m
- Merged PRs (30d)
- 48
Description
**Jetty version(s)**
Jetty 12.1.1
**Jetty Environment**
ee11
**HTTP version**
HTTP 1.1
**Java version/vendor** `(use: java -version)`
JVM : Red Hat, Inc. version: 17.0.14+7-LTS
**OS type/version**
Linux version: 4.18.0-553.44.1.el8_10.x86_64 arch: amd64
**Description**
After upgrading Apache Qpid Broker-J from Jetty 11 to Jetty 12 (currently 12.1.1), we sometimes see an uncaught `IllegalStateException: channel already completed` in a `jetty-immediate-executor` thread when HTTP connection hits the idle timeout.
Stacktrace:
```
2025-12-02T06:51:16,271Z ERROR [jetty-immediate-executor] (o.a.q.s.Main) - Uncaught exception, shutting down.
java.lang.IllegalStateException: channel already completed
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1264)
at org.eclipse.jetty.server.Response.writeError(Response.java:647)
at org.eclipse.jetty.server.Response.writeError(Response.java:550)
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
at java.base/java.lang.Thread.run(Thread.java:840)
Suppressed: java.util.concurrent.TimeoutException: Idle timeout expired: 30074/30000 ms
at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:167)
at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 common frames omitted
Suppressed: java.lang.IllegalStateException: channel already completed
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getHttpChannelState(HttpChannelState.java:864)
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getComponents(HttpChannelState.java:851)
at org.eclipse.jetty.server.Request$Wrapper.getComponents(Request.java:853)
at org.eclipse.jetty.server.handler.ErrorHandler.generateAcceptableResponse(ErrorHandler.java:203)
at org.eclipse.jetty.server.handler.ErrorHandler.generateResponse(ErrorHandler.java:163)
at org.eclipse.jetty.server.handler.ErrorHandler.handle(ErrorHandler.java:118)
at org.eclipse.jetty.server.Response.writeError(Response.java:635)
at org.eclipse.jetty.server.Response.writeError(Response.java:550)
at org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
... 1 common frames omitted
```
This looks similar in spirit to other "channel already completed" issues like #8024 or #11363, however in this case exception happens in idle-timeout / failure path while Jetty is trying to write an error.
When idle timeout fires on a connection I would expect Jetty to close / clean up the connection and optionally log an error and / or write an error response without throwing from HttpChannelState. This exception bubbles out of the Jetty executor thread (`jetty-immediate-executor`) and becomes an uncaught exception in the embedding application.
**How to reproduce?**
We've not yet managed to reduce this to a minimal standalone Jetty sample, but the issue seems to require the idle timeout firing and the associated HttpChannelState already being in "completed" state (possibly due to reuse of the underlying request / response) during the idle-timeout failure processing.
Contributor guide
Research direction
Start with HttpChannelState$ChannelResponse.write(), HttpChannelState$ChannelCallback.failed(), and the idle-timeout stack in IdleTimeout.java. Try to reduce the idle-timeout and already-completed-channel sequence to a standalone Jetty test, using the reported Jetty 12.1.1 EE11 HTTP/1.1 setup. Done means the failure path cleans up or reports the timeout without an uncaught IllegalStateException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100