aws / aws/amazon-s3-encryption-client-java

S3EncryptionClientException: Re-subscription is not supported during Multipart Upload retry (InputStream source)

未关闭
#499 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Java
星标
34
派生
21
PR 合并指标
30 天内没有已合并 PR

描述

### Problem:
We are encountering a critical failure when performing a high-level Multipart Upload (putObject) using the AWS S3 Encryption Client. The operation fails specifically during SDK automatic retry attempts triggered by network latency.

Stack trace of failure:
```
software.amazon.encryption.s3.S3EncryptionClientException: Exception while performing Multipart Upload PutObject
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:198) ~[sd-0.1.jar:?]
at ...
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[?:?]
at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[?:?]
at scala.util.Success.map(Try.scala:213) ~[?:?]
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[?:?]
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[?:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClient.onAbort(S3EncryptionClient.java:359) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:344) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: java.util.concurrent.ExecutionException: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
at software.amazon.encryption.s3.S3EncryptionClient.multipartPutObject(S3EncryptionClient.java:337) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClient.putObject(S3EncryptionClient.java:196) ~[sd-0.1.jar:?]
... 12 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:66) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.internal.UploadObjectObserver$1.call(UploadObjectObserver.java:56) ~[sd-0.1.jar:?]
at com.alibaba.ttl.TtlCallable.call(TtlCallable.java:65) ~[sd-0.1.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
... 3 more
Caused by: software.amazon.encryption.s3.S3EncryptionClientException: Re-subscription is not supported! Retry the entire operation. (SDK Attempt Count: 2)
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:67) ~[sd-0.1.jar:?]
at software.amazon.encryption.s3.S3EncryptionClientException$BuilderImpl.build(S3EncryptionClientException.java:44) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:139) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:157) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:117) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2094) ~[?:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255) ~[sd-0.1.jar:?]
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167) ~[sd-0.1.jar:?]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930) ~[?:?]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907) ~[?:?]
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) ~[?:?]
... 3 more
Suppressed: software.amazon.awssdk.core.exception.SdkClientException: Request attempt 1 failure: Unable to execute HTTP request: Write timed out
```
Environment:

Component: AWS S3 Encryption Client (High-level Multipart Upload)
SDK Version: software.amazon.encryption.s3: "3.3.5", software.amazon.awssdk: "2.31.67"
**Network Conditions: Outgoing traffic on port 443 is throttled to 1 Mbps using tc rules.**
Data Source: InputStream (Strict requirement)

Observed behaviour:
1. Due to the 1 Mbps throttle, the upload of a part occasionally times out or disconnects.
2. The AWS SDK correctly identifies the failure and attempts a retry (indicated by SDK Attempt Count: 2 in logs).
3. The retry logic fails immediately because the S3 Encryption Client is unable to reset or re-subscribe to the underlying InputStream.
4. Code snippets:
```
S3EncryotionClient:
S3EncryptionClient
.builder()
.credentialsProvider(...)
.wrappedClient(unencryptedS3Client)
.dualstackEnabled(shouldUseDualStack)
.overrideConfiguration(clientConfigs)
.enableMultipartPutObject(enableHighLevelMultipartUpload)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.enableDelayedAuthenticationMode(true)
.cryptoProvider(cryptoComplyFipsProvider)
unencryptedS3Client:
S3Client
.builder()
.credentialsProvider(
credentials.getAwsV2CredentialsProvider
)
.forcePathStyle(usePathStyleAccess)
.dualstackEnabled(shouldUseDualStack)
.httpClientBuilder(httpClient)
.overrideConfiguration(clientConfigs)
.endpointOverride(s3UriEndpoint)
.region(Region.of(s3Region))
.build()
httpClient:
ApacheHttpClient
.builder()
.useIdleConnectionReaper(useApacheIdleConnectionReaper)
.maxConnections(awsConf.awsMaxConnections.get)
.connectionTimeout(Duration.ofMillis(1000000))
.socketTimeout(Duration.ofMillis(1000000))
.connectionMaxIdleTime(Duration.ofMillis(1000000))
```

### Solution:

It seems sdk is no honouring the timeouts given in httpClient and instead sdk is internally creating a wrapped async client which have its own timeout values.
The above claim is supported by the user agent value found in multipart upload request:
```
POST https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile__metadata?uploads
User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#async md/http#NettyNio ...]

while on the other hand head request headers:

HEAD https://bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com/5gbfile
Host=[bucket-rsa-akshay-126.s3.us-west-2.amazonaws.com], User-Agent=[APN/1.0 Rubrik/1.0 Thor/1.0 aws-sdk-java/2.31.67 md/io#sync md/http#Apache...]
```

if async client is mandatory for s3 client it should be restricted if not then encryption client should always use the ApacheHttpClient.

贡献指南

打开贡献指南

调研方向

从 S3EncryptionClient.java 的第 196-198 行和第 337-359 行附近开始,然后检查 UploadObjectObserver.java 的第 56-66 行附近,以跟踪 InputStream 源的 multipart 重试。比较同步 Apache 客户端的配置与 multipart 上传所使用的异步客户端配置。当受限流或断开连接的 multipart 上传能够在不发生重新订阅失败的情况下重试 InputStream 源,并且对观察到的路径具有回归覆盖时,即视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, java
领域
cloud, security
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。