aws / aws/aws-sdk-php

Client Side Encryption with S3EncryptionMultipartUploaderV2 fails on large files.

Open
#2,512 2 comments 0 reactions 1 assignee Claimed by @stobrien89 View on GitHub
documentation p2 queued
Dominant language
PHP
Stars
6.2k
Forks
1.3k
Avg merge
2h 53m
Merged PRs (30d)
7

Description

### Describe the bug

When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB a warning is thrown during encryption, and an exception is thrown during upload. This does not happen with smaller files.

### Expected Behavior

Success on S3EncryptionMultipartUploaderV2::upload()

### Current Behavior

When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB this message appears:

`PHP Warning: openssl_encrypt(): data is too long in libs\aws\aws-sdk-php\src\Crypto\AesGcmEncryptingStream.php on line 108`

After a while, exception is thrown:

```
An exception occurred while completing a multipart upload: Error executing "CompleteMultipartUpload" on "encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4"; AWS HTTP error: Client error: `POST encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4` resulted in a `400 Bad Request` response:

MalformedXMLThe XML you provided was not well-formed (truncated...)
MalformedXML (client): The XML you provided was not well-formed or did not validate against our published schema. -
MalformedXMLThe XML you provided was not well-formed or did not validate against our published schema.lfmb_TESNICO.mp4encryption-test/encryption-test/lfmb_TESNICO.mp4170FDB5FFCA888484bf9e65b-bbb0-488d-9da9-d54f60abb5c0
```

### Reproduction Steps

```php
$uploader = new S3EncryptionMultipartUploaderV2(
new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'use_path_style_endpoint' => true,
'credentials' => [
'key' => $options['credentials']['key'],
'secret' => $options['credentials']['secret'],
]
]), ,fopen('path to a file larger than 4GB', 'r'),
[
'@MaterialsProvider' => $materialsProvider,
'@CipherOptions' => [ 'Cipher' => 'gcm', 'KeySize' => 128 ],
'@KmsEncryptionContext' => [],
'bucket' => $this->bucket,
'key' => 'thenamefile',
'concurrency' => 5,
'part_size' => null,
]
);

$uploader->upload();

```

### Possible Solution

openssl_encrypt is know have trouble with large files.
Maybe include a mechanism to encode by chunks.

### Additional Information/Context

_No response_

### SDK version used

"aws/aws-sdk-php": "^3.147"

### Environment details (Version of PHP (`php -v`)? OS name and version, etc.)

PHP 7.4.30 (cli) (built: Jun 7 2022 15:36:03) ( NTS Visual C++ 2017 x64 )

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.