aws / aws/aws-sdk-java-v2

RetryOnException in RetryStrategy does not retry on UncheckedIOException

Open
#6,578 0 comments 0 reactions 1 assignee Claimed by @debora-ito View on GitHub
bug needs-review p2
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

### Describe the bug

Customizing retryable exceptions as described in the [documentation](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/retry-strategy.html) by using the `retryOnException` function in `RetryStrategy` class does not trigger retries when an `UncheckedIOException` is configured.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

The retry strategy should recognise `UncheckedIOException` as an additional retryable exception and automatically retry operations when this exception is thrown.

### Current Behavior

Stack trace from the local test using TestContainers:
```
java.io.UncheckedIOException: java.io.IOException
at com.qmetric.testcontainersutils.TestHttpClient$1.call(S3ClientTest.java:60)
at com.qmetric.testcontainersutils.TestHttpClient$1.call(S3ClientTest.java:50)
at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:103)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:88)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:64)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:46)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:79)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:41)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest(RetryableStage.java:93)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:56)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:53)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:35)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:82)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:62)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:210)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:12535)
at com.example.S3ClientTest.testS3Client(S3ClientTest.java:32)
```

Stack trace from the issue in production:
```
software.amazon.awssdk.utils.FunctionalUtils.asRuntimeException (FunctionalUtils.java:180)
software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4 (FunctionalUtils.java:110)
software.amazon.awssdk.utils.FunctionalUtils.invokeSafely (FunctionalUtils.java:136)
software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.lambda$tryGetOutputStream$0 (UrlConnectionHttpClient.java:331)
software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.getAndHandle100Bug (UrlConnectionHttpClient.java:367)
software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.tryGetOutputStream (UrlConnectionHttpClient.java:331)
software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.call (UrlConnectionHttpClient.java:308)
software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.call (UrlConnectionHttpClient.java:281)
software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe (MetricUtils.java:103)
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest (MakeHttpRequestStage.java:88)
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute (MakeHttpRequestStage.java:64)
software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute (MakeHttpRequestStage.java:46)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute (ApiCallAttemptTimeoutTrackingStage.java:74)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute (ApiCallAttemptTimeoutTrackingStage.java:43)
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute (TimeoutExceptionHandlingStage.java:79)
software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute (TimeoutExceptionHandlingStage.java:41)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute (ApiCallAttemptMetricCollectionStage.java:55)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute (ApiCallAttemptMetricCollectionStage.java:39)
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.executeRequest (RetryableStage.java:93)
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute (RetryableStage.java:56)
software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute (RetryableStage.java:36)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute (StreamManagingStage.java:53)
software.amazon.awssdk.core.internal.http.StreamManagingStage.execute (StreamManagingStage.java:35)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer (ApiCallTimeoutTrackingStage.java:82)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute (ApiCallTimeoutTrackingStage.java:62)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute (ApiCallTimeoutTrackingStage.java:43)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute (ApiCallMetricCollectionStage.java:50)
software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute (ApiCallMetricCollectionStage.java:32)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute (RequestPipelineBuilder.java:206)
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute (ExecutionFailureExceptionReportingStage.java:37)
software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute (ExecutionFailureExceptionReportingStage.java:26)
software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute (AmazonSyncHttpClient.java:210)
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke (BaseSyncClientHandler.java:103)
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute (BaseSyncClientHandler.java:173)
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1 (BaseSyncClientHandler.java:80)
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess (BaseSyncClientHandler.java:182)
software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute (BaseSyncClientHandler.java:74)
software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute (SdkSyncClientHandler.java:45)
software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute (AwsSyncClientHandler.java:53)
software.amazon.awssdk.services.s3.DefaultS3Client.putObject (DefaultS3Client.java:11883)
com.example.S3ObjectStoreKt$s3$1.write (S3ObjectStore.kt:26)
com.example.S3Store.store (S3Store.java:50)
com.example.S3Store.lambda$requestProcessed$0 (S3Store.java:42)
io.opentelemetry.javaagent.bootstrap.executors.ContextPropagatingRunnable.run (ContextPropagatingRunnable.java:37)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
java.lang.Thread.run (Thread.java:1583)
```

### Reproduction Steps

```
import org.junit.Assert;
import org.junit.Test;
import org.testcontainers.containers.localstack.LocalStackContainer;
import org.testcontainers.utility.DockerImageName;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.http.ExecutableHttpRequest;
import software.amazon.awssdk.http.HttpExecuteRequest;
import software.amazon.awssdk.http.HttpExecuteResponse;
import software.amazon.awssdk.http.SdkHttpClient;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import java.io.IOException;
import java.io.UncheckedIOException;

public class S3ClientTest {
@Test
public void testS3Client() {
TestHttpClient httpClient = new TestHttpClient();
try (LocalStackContainer localStack = new LocalStackContainer(DockerImageName.parse("localstack/localstack"))
.withServices(LocalStackContainer.Service.S3)) {
localStack.start();
try (S3Client s3Client = S3Client.builder()
.credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(localStack.getAccessKey(), localStack.getSecretKey())))
.endpointOverride(localStack.getEndpointOverride(LocalStackContainer.Service.S3))
.httpClient(httpClient)
.overrideConfiguration(o -> o.retryStrategy(b -> b.retryOnException(UncheckedIOException.class)))
.build()) {
s3Client.putObject(
PutObjectRequest.builder().bucket("test-bucket").key("test-key").build(),
RequestBody.fromString("test-content")
);
} catch (Exception e) {
e.printStackTrace();
}
}
Assert.assertEquals(4, httpClient.count);
}
}

class TestHttpClient implements SdkHttpClient {

public int count = 0;

@Override
public ExecutableHttpRequest prepareRequest(HttpExecuteRequest httpExecuteRequest) {
return new ExecutableHttpRequest() {

@Override
public void abort() {
}

@Override
public HttpExecuteResponse call() throws IOException {
count++;
throw new UncheckedIOException(new IOException());
}
};
}

@Override
public void close() {
}
}
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.37.3 and 2.39.0

### JDK version used

openjdk 21.0.3 2024-04-16 LTS

### Operating System and version

alpine 3.19

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.