(aws-s3-deployment/BucketDeployment): Deployment failed if file greater than 500mb
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
I believe this bug is related to the lambda limitation of the storage size of temp directory (/tmp)
Deployment will be failed if file greater than 500mb.
### Reproduction Steps
### What did you expect to happen?
Support arbitrary size of the file. Since the file is uploaded to s3, it should be able to move it to another s3 when deploy.
### What actually happened?
Deployment failed if file greater than 500mb
1:59:56 PM | CREATE_FAILED | Custom::CDKBucketDeployment | s3uploadAssetsToS3CustomResource6A20AD81
Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdk-hnb659fds-assets-***-us-east-1/assets/e
f19b4b88c51ddd1c4f0445c48aed9f9dfd081bf64bed56c8735308a0a1c2ddf.zip', '/tmp/tmp1vubm6v5/537b3088-e250-49ec-9f95-813c1f4b96e1']' returned non-zero exit status 1. (RequestId:
468e1fe9-8c73-4883-a321-11bc6fd63935)
https://github.com/aws/aws-cdk/blob/c92548b2242478d22db030842014e7646715c2ef/packages/%40aws-cdk/aws-s3-deployment/lib/lambda/index.py#L130
### Environment
- **CDK CLI Version : 1.115.0**
- **Framework Version: 1.115.0**
- **Node.js Version: v14.16.0**
- **OS : Mac OS **
- **Language (Version): Javascript **
### Other
---
This is :bug: Bug Report
Contributor guide
Research direction
Start by reading packages/@aws-cdk/aws-s3-deployment/lib/lambda/index.py around line 130, then reproduce the reported failure with an asset larger than 500 MB. Check how the deployment handles the temporary download and confirm the completed behavior by deploying a file above that size without exceeding Lambda's temporary storage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100