aws / aws/aws-cdk

(aws-s3-deployment/BucketDeployment): Deployment failed if file greater than 500mb

Open
#15,723 5 comments 1 reaction 0 assignees View on GitHub
@aws-cdk/aws-s3-deployment bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

I believe this bug is related to the lambda limitation of the storage size of temp directory (/tmp)
Deployment will be failed if file greater than 500mb.

### Reproduction Steps

### What did you expect to happen?
Support arbitrary size of the file. Since the file is uploaded to s3, it should be able to move it to another s3 when deploy.

### What actually happened?
Deployment failed if file greater than 500mb

1:59:56 PM | CREATE_FAILED | Custom::CDKBucketDeployment | s3uploadAssetsToS3CustomResource6A20AD81
Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdk-hnb659fds-assets-***-us-east-1/assets/e
f19b4b88c51ddd1c4f0445c48aed9f9dfd081bf64bed56c8735308a0a1c2ddf.zip', '/tmp/tmp1vubm6v5/537b3088-e250-49ec-9f95-813c1f4b96e1']' returned non-zero exit status 1. (RequestId:
468e1fe9-8c73-4883-a321-11bc6fd63935)

https://github.com/aws/aws-cdk/blob/c92548b2242478d22db030842014e7646715c2ef/packages/%40aws-cdk/aws-s3-deployment/lib/lambda/index.py#L130

### Environment

- **CDK CLI Version : 1.115.0**
- **Framework Version: 1.115.0**
- **Node.js Version: v14.16.0**
- **OS : Mac OS **
- **Language (Version): Javascript **

### Other

---

This is :bug: Bug Report

Contributor guide

Open the contributing guide

Research direction

Start by reading packages/@aws-cdk/aws-s3-deployment/lib/lambda/index.py around line 130, then reproduce the reported failure with an asset larger than 500 MB. Check how the deployment handles the temporary download and confirm the completed behavior by deploying a file above that size without exceeding Lambda's temporary storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.