humanmade / humanmade/S3-Uploads

0 byte file version created when object versioning is enabled

Open
#409 2 comments 0 reactions 0 assignees View on GitHub
Bug
Dominant language
PHP
Stars
2.2k
Forks
404
PR merge metrics
No merged PRs in 30d

Description

When versioning is enabled on an S3 bucket (also confirmed with other cloud providers), the plugin creates 2 versions of each file as they are uploaded. It first creates a 0 byte version of the file, then uploads the real version. Both versions are left in S3. This is with an access key with full S3 global permissions. Here is an excerpt from the AWS debug log:

First empty file:
> > PUT /1/1/uploads/2020/05/ds_beach.jpg HTTP/1.1
> Host: iu-test.s3.us-east-2.amazonaws.com
> Content-Type: application/octet-stream
> content-length: 0
> aws-sdk-invocation-id: ec7a1c6e2a93e986b49fb5007b670247
> aws-sdk-retry: 0/0
> X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> X-Amz-Date: 20200515T163312Z
> Authorization: AWS4-HMAC-SHA256 Credential=[KEY]/20200515/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=[SIGNATURE]
> User-Agent: aws-sdk-php/3.137.6 GuzzleHttp/6.5.3 curl/7.58.0 PHP/7.2.26-1+ubuntu18.04.1+deb.sury.org+1
>
> < HTTP/1.1 200 OK
> < x-amz-id-2: WbRUb61CIbhhZ5+4uYLTH+doizWUlbx4DueRIwuHl7BvBTXE/sGoFygMmzJHoJMlqnNbA7ixafA=
> < x-amz-request-id: 162D6F78684555C7
> < Date: Fri, 15 May 2020 16:33:13 GMT
> < x-amz-version-id: JMyHUxSTJDDIYrgVWrZJB1LIjTMrg9rX
> < ETag: "d41d8cd98f00b204e9800998ecf8427e"
> < Content-Length: 0
> < Server: AmazonS3

Then full file:
> > PUT /1/1/uploads/2020/05/ds_beach.jpg HTTP/1.1
> Host: iu-test.s3.us-east-2.amazonaws.com
> Expect: 100-Continue
> x-amz-acl: public-read
> Content-Type: image/jpeg
> Cache-Control: max-age=31536000
> aws-sdk-invocation-id: c078a8391b973af4d30c4dfa7e589307
> aws-sdk-retry: 0/0
> X-Amz-Content-Sha256: 387b2b7be6db9b1dad1ad9d0b5cff4b79527064a1e63e9515e82f952f7b1693f
> X-Amz-Date: 20200515T163312Z
> Authorization: AWS4-HMAC-SHA256 Credential=[KEY]/20200515/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=[SIGNATURE]
> User-Agent: aws-sdk-php/3.137.6 GuzzleHttp/6.5.3 curl/7.58.0 PHP/7.2.26-1+ubuntu18.04.1+deb.sury.org+1
> Content-Length: 1739289
>
> < HTTP/1.1 100 Continue
> * We are completely uploaded and fine
> < HTTP/1.1 200 OK
> < x-amz-id-2: YgLNX76GpTxwTjFHIqXDRMDI5XarGvt7g3Jb8F0N5ur85kwZho79a3s5ff/+PceAHeK9AMA8yE0=
> < x-amz-request-id: 2D8ED48297DCEA21
> < Date: Fri, 15 May 2020 16:33:13 GMT
> < x-amz-version-id: 1Hot9Ie64mxQWaWfMExwURABhSanOZtD
> < ETag: "53c22d30b59e9e759981aaa86d9b27e8"
> < Content-Length: 0
> < Server: AmazonS3

I don't think this is expected behavior, and even with versioning turned off would be a performance and cost impact as it has to make 2 PUT api calls for every object.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.