aws / aws/amazon-s3-encryption-client-java

S3EncryptionClientException: Re-subscription is not supported during Multipart Upload retry (InputStream source)

Offen
#499 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Java
Sterne
34
Forks
21
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

### Problem:
We are encountering a critical failure when performing a high-level Multipart Upload (putObject) using the AWS S3 Encryption Client. The operation fails specifically during SDK automatic retry attempts triggered by network latency.

Stack trace of failure:
```
software.amazon.encryption.s3.S3EncryptionClientException: Exception while performing Multipart Upload PutObject
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:198) ~[sd-0.1.jar:?]
at ...
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[?:?]
at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[?:?]
at scala.util.Success.map(Try.scala:213) ~[?:?]
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[?:?]
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClient.onAbort(S3EncryptionClient.java:359) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:344) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: java.util.concurrent.ExecutionException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:337) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:66) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:56) ~[sd-0.1.jar:?]
at com.alibaba.ttl.TtlCallable.call(TtlCallable.java:65) ~[sd-0.1.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
... 3 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:67) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:44) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:139) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:157) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:117) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930) ~[?:?]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907) ~[?:?]
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) ~[?:?]
... 3 more
Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1 failure: Unable to execute HTTP request: Write timed out
```
Environment:

Component: AWS S3 Encryption Client (High-level Multipart Upload)
SDK Version: software.amazon.encryption.s3: "3.3.5", software.amazon.awssdk: "2.31.67"
**Network Conditions: Outgoing traffic on port 443 is throttled to 1 Mbps using tc rules.**
Data Source: InputStream (Strict requirement)

Observed behaviour:
1. Due to the 1 Mbps throttle, the upload of a part occasionally times out or disconnects.
2. The AWS SDK correctly identifies the failure and attempts a retry (indicated by SDK Attempt Count: 2 in logs).
3. The retry logic fails immediately because the S3 Encryption Client is unable to reset or re-subscribe to the underlying InputStream.
4. Code snippets:
```
S3EncryotionClient:
S3EncryptionClient
.builder()
.credentialsProvider(...)
.wrappedClient(unencryptedS3Client)
.dualstackEnabled(shouldUseDualStack)
.overrideConfiguration(clientConfigs)
.enableMultipartPutObject(enableHighLevelMultipartUpload)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.enableDelayedAuthenticationMode(true)
.cryptoProvider(cryptoComplyFipsProvider)
unencryptedS3Client:
S3Client
.builder()
.credentialsProvider(
credentials.getAwsV2CredentialsProvider
)
.forcePathStyle(usePathStyleAccess)
.dualstackEnabled(shouldUseDualStack)
.httpClientBuilder(httpClient)
.overrideConfiguration(clientConfigs)
.endpointOverride(s3UriEndpoint)
.region(Region.of(s3Region))
.build()
httpClient:
ApacheHttpClient
.builder()
.useIdleConnectionReaper(useApacheIdleConnectionReaper)
.maxConnections(awsConf.awsMaxConnections.get)
.connectionTimeout(Duration.ofMillis(1000000))
.socketTimeout(Duration.ofMillis(1000000))
.connectionMaxIdleTime(Duration.ofMillis(1000000))
```

### Solution:

It seems sdk is no honouring the timeouts given in httpClient and instead sdk is internally creating a wrapped async client which have its own timeout values.
The above claim is supported by the user agent value found in multipart upload request:
```
POST https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile__metadata?uploads
User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#async md/http#NettyNio ...]

while on the other hand head request headers:

HEAD https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile
Host=[bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com], User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#sync md/http#Apache...]
```

if async client is mandatory for s3 client it should be restricted if not then encryption client should always use the ApacheHttpClient.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit S3EncryptionClient.java ungefähr in den Zeilen 196–198 und 337–359 und untersuche anschließend UploadObjectObserver.java ungefähr in den Zeilen 56–66, um Multipart-Wiederholungen für InputStream-Quellen nachzuverfolgen. Vergleiche die Konfiguration des synchronen Apache-Clients mit dem asynchronen Client, der beim Multipart-Upload verwendet wird. Als abgeschlossen gilt die Arbeit, wenn ein gedrosselter oder unterbrochener Multipart-Upload eine InputStream-Quelle ohne den Fehler bei der erneuten Subskription wiederholen kann und eine Regressionstestabdeckung für den beobachteten Pfad vorhanden ist.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, java
Bereich
cloud, security
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.