aws / aws/amazon-s3-encryption-client-java
Multipart upload with CSE using RSA key pair
- Lingua principale
- Java
- Stelle
- 34
- Fork
- 21
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
### Problem:
My code is roughly
```
val s3ClientObject =
S3Client
.builder()
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(
spec.accessKey.get,
getSecretKey(metadataEncryptionUtils)
)
)
)
.region(REGION.US_EAST_1).build()
val s3AsyncClientObject =
S3AsyncClient
.builder()
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(
spec.accessKey.get,
getSecretKey(metadataEncryptionUtils)
)
)
)
.region(REGION.US_EAST_1).build()
```
Now I create S3EncryptionClient by wrapping above such as
```
val encObject =
S3EncryptionClient
.builder()
.rsaKeyPair(userKeys)
.enableLegacyUnauthenticatedModes(true)
.enableLegacyWrappingAlgorithms(true)
.wrappedClient(s3ClientObject)
.wrappedAsyncClient(s3AsyncClientObject)
.enableDelayedAuthenticationMode(true)
.build()
```
I am able to use this `encObject` to do operations like creating bucket etc. I am also able to upload files to s3 bucket.
However, when i try to upload a large file ( say ~200MB ) with multi part upload it fails with following error
```
aused by: software.amazon.awssdk.crt.http.HttpException: Amount of data streamed out does not match the previously declared length.
at software.amazon.awssdk.http.crt.internal.response.CrtResponseAdapter.onResponseComplete(CrtResponseAdapter.java:108) ~[thirdparty-intellij-deps.jar:?]
at software.amazon.awssdk.crt.http.HttpStreamResponseHandlerNativeAdapter.onResponseComplete(HttpStreamResponseHandlerNativeAdapter.java:58) ~[thirdparty-intellij-deps.jar:?]
Exception in thread "AwsEventLoop 9" java.lang.IllegalStateException: Encountered fatal error in publisher
at software.amazon.awssdk.utils.async.SimplePublisher.panicAndDie(SimplePublisher.java:339)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:226)
at software.amazon.awssdk.utils.async.SimplePublisher.send(SimplePublisher.java:128)
at software.amazon.awssdk.utils.async.InputStreamConsumingPublisher.doBlockingWrite(InputStreamConsumingPublisher.java:58)
at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.writeInputStream(BlockingInputStreamAsyncRequestBody.java:76)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.doBlockingWrite(InputStreamWithExecutorAsyncRequestBody.java:108)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.lambda$subscribe$0(InputStreamWithExecutorAsyncRequestBody.java:81)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: Must use either different key or iv for GCM encryption
at java.base/com.sun.crypto.provider.CipherCore.checkReinit(CipherCore.java:1088)
at java.base/com.sun.crypto.provider.CipherCore.update(CipherCore.java:662)
at java.base/com.sun.crypto.provider.AESCipher.engineUpdate(AESCipher.java:380)
at java.base/javax.crypto.Cipher.update(Cipher.java:1869)
at software.amazon.encryption.s3.internal.CipherSubscriber.onNext(CipherSubscriber.java:52)
at software.amazon.encryption.s3.internal.CipherSubscriber.onNext(CipherSubscriber.java:16)
at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:267)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224)
... 10 more
```
If I directly use the `s3ClientObject` it works.
### Solution:
Is there some limitation on CSE with multi part uploads ?
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Reproduce the roughly 200 MB multipart upload using the S3EncryptionClient builder with rsaKeyPair, then compare it with the direct s3ClientObject path. Start from the CipherSubscriber stack trace and multipart upload flow; done means encrypted multipart uploads complete without the declared-length or GCM key/IV errors.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, java
- Ambito
- backend-api-design, security
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100