aws / aws/amazon-s3-encryption-client-java

Multipart upload with CSE using RSA key pair

オープン
#268 コメント 5 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Java
スター
34
フォーク
21
PR マージ指標
30日以内にマージされた PR はありません

説明

### Problem:

My code is roughly
```
val s3ClientObject =
S3Client
.builder()
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(
spec.accessKey.get,
getSecretKey(metadataEncryptionUtils)
)
)
)
.region(REGION.US_EAST_1).build()

val s3AsyncClientObject =
S3AsyncClient
.builder()
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(
spec.accessKey.get,
getSecretKey(metadataEncryptionUtils)
)
)
)
.region(REGION.US_EAST_1).build()
```
Now I create S3EncryptionClient by wrapping above such as

```
val encObject =
S3EncryptionClient
.builder()
.rsaKeyPair(userKeys)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.wrappedClient(s3ClientObject)
.wrappedAsyncClient(s3AsyncClientObject)
.enableDelayedAuthenticationMode(true)
.build()
```

I am able to use this `encObject` to do operations like creating bucket etc. I am also able to upload files to s3 bucket.

However, when i try to upload a large file ( say ~200MB ) with multi part upload it fails with following error

```
aused by: software.amazon.awssdk.crt.http.HttpException: Amount of data streamed out does not match the previously declared length.
at software.amazon.awssdk.http.crt.internal.response.CrtResponseAdapter.onResponseComplete(CrtResponseAdapter.java:108) ~[thirdparty-intellij-deps.jar:?]
at software.amazon.awssdk.crt.http.HttpStreamResponseHandlerNativeAdapter.onResponseComplete(HttpStreamResponseHandlerNativeAdapter.java:58) ~[thirdparty-intellij-deps.jar:?]
Exception in thread "AwsEventLoop 9" java.lang.IllegalStateException: Encountered fatal error in publisher
at software.amazon.awssdk.utils.async.SimplePublisher.panicAndDie(SimplePublisher.java:339)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:226)
at software.amazon.awssdk.utils.async.SimplePublisher.send(SimplePublisher.java:128)
at software.amazon.awssdk.utils.async.InputStreamConsumingPublisher.doBlockingWrite(InputStreamConsumingPublisher.java:58)
at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.writeInputStream(BlockingInputStreamAsyncRequestBody.java:76)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.doBlockingWrite(InputStreamWithExecutorAsyncRequestBody.java:108)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.lambda$subscribe$0(InputStreamWithExecutorAsyncRequestBody.java:81)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: Must use either different key or iv for GCM encryption
at java.base/com.sun.crypto.provider.CipherCore.checkReinit(CipherCore.java:1088)
at java.base/com.sun.crypto.provider.CipherCore.update(CipherCore.java:662)
at java.base/com.sun.crypto.provider.AESCipher.engineUpdate(AESCipher.java:380)
at java.base/javax.crypto.Cipher.update(Cipher.java:1869)
at software.amazon.encryption.s3.internal.CipherSubscriber.onNext(CipherSubscriber.java:52)
at software.amazon.encryption.s3.internal.CipherSubscriber.onNext(CipherSubscriber.java:16)
at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:267)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224)
... 10 more
```

If I directly use the `s3ClientObject` it works.

### Solution:

Is there some limitation on CSE with multi part uploads ?

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

rsaKeyPair を指定した S3EncryptionClient builder を使用して約 200 MB の multipart upload を再現し、その後、直接の s3ClientObject パスと比較します。CipherSubscriber の stack trace と multipart upload のフローから開始します。暗号化された multipart upload が declared-length または GCM key/IV エラーなしで完了すれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, java
領域
backend-api-design, security
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。