eclipse-ee4j / eclipse-ee4j/jersey
WritePendingException on async socket write
- Dominant language
- Java
- Stars
- 730
- Forks
- 382
- PR merge metrics
- No merged PRs in 30d
Description
I'm seeing an occasional Jersey ProcessingException with embedded WritePendingException when trying to send a request:
```
java.io.IOException: Reserve key failed for [366655488-370849791] - RESERVING
at com.primarydata.dme.mover.steps.ReserveKeyRunnable$1.failed(ReserveKeyRunnable.java:103) ~[mover-core-4.2.0.jar:?]
at com.hammerspace.common.utils.OutstandingRequestTrackingHandlerImpl.failed(OutstandingRequestTrackingHandlerImpl.java:79) ~[dme-common-4.2.0.jar:?]
at org.glassfish.jersey.client.JerseyInvocation$4.failed(JerseyInvocation.java:1030) ~[jersey-client-2.26.jar:?]
at org.glassfish.jersey.client.ClientRuntime.processFailure(ClientRuntime.java:231) ~[jersey-client-2.26.jar:?]
at org.glassfish.jersey.client.ClientRuntime.lambda$null$3(ClientRuntime.java:189) ~[jersey-client-2.26.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:316) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:298) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:268) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:312) ~[jersey-common-2.26.jar:?]
at org.glassfish.jersey.client.ClientRuntime.lambda$createRunnableForAsyncProcessing$4(ClientRuntime.java:163) ~[jersey-client-2.26.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: javax.ws.rs.ProcessingException: java.nio.channels.WritePendingException
... 14 more
Caused by: java.nio.channels.WritePendingException
at sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:352) ~[?:1.8.0_181]
at sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousSocketChannelImpl.java:399) ~[?:1.8.0_181]
at java.nio.channels.AsynchronousSocketChannel.write(AsynchronousSocketChannel.java:577) ~[?:1.8.0_181]
at org.glassfish.jersey.jdk.connector.internal.TransportFilter.write(TransportFilter.java:127) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.TransportFilter.write(TransportFilter.java:76) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpFilter.write(HttpFilter.java:71) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpFilter.write(HttpFilter.java:51) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpConnection$ConnectionFilter.write(HttpConnection.java:453) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpConnection$ConnectionFilter.write(HttpConnection.java:365) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpConnection.send(HttpConnection.java:141) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.DestinationConnectionPool.processPendingRequests(DestinationConnectionPool.java:143) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.DestinationConnectionPool.send(DestinationConnectionPool.java:89) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.HttpConnectionPool.send(HttpConnectionPool.java:98) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.JdkConnector.send(JdkConnector.java:173) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.jdk.connector.internal.JdkConnector.apply(JdkConnector.java:133) ~[jersey-jdk-connector-2.26.jar:?]
at org.glassfish.jersey.client.ClientRuntime.lambda$null$3(ClientRuntime.java:187) ~[jersey-client-2.26.jar:?]
... 12 more
```
I've done some basic research on WritePendingException - Oracle docs indicate it's an "Unchecked exception thrown when an attempt is made to write to an asynchronous socket channel and a previous write has not completed." This seems too low-level for a jersey client user to have done anything to cause it. Is this a problem with Jersey?
I'm using Jersey 2.26 and the call being made is using async client methods like this:
```
try {
Future future = client.target(baseUri)
.path(KEYS)
.path("{key}").resolveTemplate("key", DatatypeConverter.printHexBinary(keyb))
.request(MediaType.APPLICATION_JSON)
.async()
.put(Entity.entity(keyRequest, MediaType.APPLICATION_JSON), handler);
handler.trackStart(future, TIMEOUT_MILLIS);
} catch (Throwable t) {
callerHandler.failed(t);
}
```
Note we're using the JdkConnectorProvider. This was done to solve an earlier issue that turned out not to be related to the connector provider, so we could switch back to the default connector provider if you believe this might be the cause of the problem. We're creating the client in this manner:
```
ClientBuilder builder = ClientBuilder.newBuilder();
this.client = builder
.executorService(es)
.register(JsonProcessingFeature.class)
.register(LoggingFeature.class)
.property(LoggingFeature.LOGGING_FEATURE_LOGGER_NAME_CLIENT, log.getName())
.connectTimeout(TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)
.readTimeout(TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)
.withConfig(new ClientConfig().connectorProvider(new JdkConnectorProvider()))
.build();
```
The same client is used by multiple threads simultaneously and without any sort of synchronization - my understanding is that Jersey clients are thread-safe and do not require sync.
This is very intermittent - it only happens under heavy load and even then only occasionally.
Thanks in advance,
John
Contributor guide
Assessment
This issue has not been assessed yet.