aws / aws/amazon-s3-encryption-client-java
S3EncryptionClientException: Re-subscription is not supported during Multipart Upload retry (InputStream source)
- Lenguaje dominante
- Java
- Estrellas
- 34
- Forks
- 21
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
### Problem:
We are encountering a critical failure when performing a high-level Multipart Upload (putObject) using the AWS S3 Encryption Client. The operation fails specifically during SDK automatic retry attempts triggered by network latency.
Stack trace of failure:
```
software.amazon.encryption.s3.S3EncryptionClientException: Exception while performing Multipart Upload PutObject
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:198) ~[sd-0.1.jar:?]
at ...
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[?:?]
at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[?:?]
at scala.util.Success.map(Try.scala:213) ~[?:?]
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[?:?]
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClient.onAbort(S3EncryptionClient.java:359) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:344) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: java.util.concurrent.ExecutionException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:337) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:66) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:56) ~[sd-0.1.jar:?]
at com.alibaba.ttl.TtlCallable.call(TtlCallable.java:65) ~[sd-0.1.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
... 3 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:67) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:44) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:139) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:157) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:117) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930) ~[?:?]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907) ~[?:?]
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) ~[?:?]
... 3 more
Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1 failure: Unable to execute HTTP request: Write timed out
```
Environment:
Component: AWS S3 Encryption Client (High-level Multipart Upload)
SDK Version: software.amazon.encryption.s3: "3.3.5", software.amazon.awssdk: "2.31.67"
**Network Conditions: Outgoing traffic on port 443 is throttled to 1 Mbps using tc rules.**
Data Source: InputStream (Strict requirement)
Observed behaviour:
1. Due to the 1 Mbps throttle, the upload of a part occasionally times out or disconnects.
2. The AWS SDK correctly identifies the failure and attempts a retry (indicated by SDK Attempt Count: 2 in logs).
3. The retry logic fails immediately because the S3 Encryption Client is unable to reset or re-subscribe to the underlying InputStream.
4. Code snippets:
```
S3EncryotionClient:
S3EncryptionClient
.builder()
.credentialsProvider(...)
.wrappedClient(unencryptedS3Client)
.dualstackEnabled(shouldUseDualStack)
.overrideConfiguration(clientConfigs)
.enableMultipartPutObject(enableHighLevelMultipartUpload)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.enableDelayedAuthenticationMode(true)
.cryptoProvider(cryptoComplyFipsProvider)
unencryptedS3Client:
S3Client
.builder()
.credentialsProvider(
credentials.getAwsV2CredentialsProvider
)
.forcePathStyle(usePathStyleAccess)
.dualstackEnabled(shouldUseDualStack)
.httpClientBuilder(httpClient)
.overrideConfiguration(clientConfigs)
.endpointOverride(s3UriEndpoint)
.region(Region.of(s3Region))
.build()
httpClient:
ApacheHttpClient
.builder()
.useIdleConnectionReaper(useApacheIdleConnectionReaper)
.maxConnections(awsConf.awsMaxConnections.get)
.connectionTimeout(Duration.ofMillis(1000000))
.socketTimeout(Duration.ofMillis(1000000))
.connectionMaxIdleTime(Duration.ofMillis(1000000))
```
### Solution:
It seems sdk is no honouring the timeouts given in httpClient and instead sdk is internally creating a wrapped async client which have its own timeout values.
The above claim is supported by the user agent value found in multipart upload request:
```
POST https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile__metadata?uploads
User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#async md/http#NettyNio ...]
while on the other hand head request headers:
HEAD https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile
Host=[bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com], User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#sync md/http#Apache...]
```
if async client is mandatory for s3 client it should be restricted if not then encryption client should always use the ApacheHttpClient.
Guía de contribución
Línea de trabajo
Comienza con S3EncryptionClient.java alrededor de las líneas 196-198 y 337-359; después, inspecciona UploadObjectObserver.java alrededor de las líneas 56-66 para rastrear los reintentos de multipart para fuentes InputStream. Compara la configuración del cliente Apache síncrono con la del cliente asíncrono utilizado por la carga multipart. El trabajo estará terminado cuando una carga multipart limitada o desconectada pueda reintentar una fuente InputStream sin el fallo de re-suscripción, con cobertura de regresión para la ruta observada.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- aws, java
- Área
- cloud, security
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100