aws / aws/amazon-s3-encryption-client-java

S3EncryptionClientException: Re-subscription is not supported during Multipart Upload retry (InputStream source)

オープン
#499 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Java
スター
34
フォーク
21
PR マージ指標
30日以内にマージされた PR はありません

説明

### Problem:
We are encountering a critical failure when performing a high-level Multipart Upload (putObject) using the AWS S3 Encryption Client. The operation fails specifically during SDK automatic retry attempts triggered by network latency.

Stack trace of failure:
```
software.amazon.encryption.s3.S3EncryptionClientException: Exception while performing Multipart Upload PutObject
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:198) ~[sd-0.1.jar:?]
at ...
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[?:?]
at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[?:?]
at scala.util.Success.map(Try.scala:213) ~[?:?]
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[?:?]
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClient.onAbort(S3EncryptionClient.java:359) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:344) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: java.util.concurrent.ExecutionException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:337) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:66) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:56) ~[sd-0.1.jar:?]
at com.alibaba.ttl.TtlCallable.call(TtlCallable.java:65) ~[sd-0.1.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
... 3 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:67) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:44) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:139) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:157) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:117) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930) ~[?:?]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907) ~[?:?]
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) ~[?:?]
... 3 more
Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1 failure: Unable to execute HTTP request: Write timed out
```
Environment:

Component: AWS S3 Encryption Client (High-level Multipart Upload)
SDK Version: software.amazon.encryption.s3: "3.3.5", software.amazon.awssdk: "2.31.67"
**Network Conditions: Outgoing traffic on port 443 is throttled to 1 Mbps using tc rules.**
Data Source: InputStream (Strict requirement)

Observed behaviour:
1. Due to the 1 Mbps throttle, the upload of a part occasionally times out or disconnects.
2. The AWS SDK correctly identifies the failure and attempts a retry (indicated by SDK Attempt Count: 2 in logs).
3. The retry logic fails immediately because the S3 Encryption Client is unable to reset or re-subscribe to the underlying InputStream.
4. Code snippets:
```
S3EncryotionClient:
S3EncryptionClient
.builder()
.credentialsProvider(...)
.wrappedClient(unencryptedS3Client)
.dualstackEnabled(shouldUseDualStack)
.overrideConfiguration(clientConfigs)
.enableMultipartPutObject(enableHighLevelMultipartUpload)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.enableDelayedAuthenticationMode(true)
.cryptoProvider(cryptoComplyFipsProvider)
unencryptedS3Client:
S3Client
.builder()
.credentialsProvider(
credentials.getAwsV2CredentialsProvider
)
.forcePathStyle(usePathStyleAccess)
.dualstackEnabled(shouldUseDualStack)
.httpClientBuilder(httpClient)
.overrideConfiguration(clientConfigs)
.endpointOverride(s3UriEndpoint)
.region(Region.of(s3Region))
.build()
httpClient:
ApacheHttpClient
.builder()
.useIdleConnectionReaper(useApacheIdleConnectionReaper)
.maxConnections(awsConf.awsMaxConnections.get)
.connectionTimeout(Duration.ofMillis(1000000))
.socketTimeout(Duration.ofMillis(1000000))
.connectionMaxIdleTime(Duration.ofMillis(1000000))
```

### Solution:

It seems sdk is no honouring the timeouts given in httpClient and instead sdk is internally creating a wrapped async client which have its own timeout values.
The above claim is supported by the user agent value found in multipart upload request:
```
POST https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile__metadata?uploads
User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#async md/http#NettyNio ...]

while on the other hand head request headers:

HEAD https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile
Host=[bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com], User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#sync md/http#Apache...]
```

if async client is mandatory for s3 client it should be restricted if not then encryption client should always use the ApacheHttpClient.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

まず S3EncryptionClient.java の 196-198 行付近と 337-359 行付近から始め、次に UploadObjectObserver.java の 56-66 行付近を調べて、InputStream ソースの multipart リトライを追跡します。同期 Apache クライアントの設定と、multipart アップロードで使用される非同期クライアントを比較します。スロットリングされた、または切断された multipart アップロードで、再サブスクライブの失敗を起こさずに InputStream ソースをリトライでき、観測されたパスに対するリグレッションカバレッジがあれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, java
領域
cloud, security
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。