(s3-deployment): Support large s3 bucket deployments
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 74
Description
We want to enable users to deploy large amounts of data onto s3 buckets. Currently, such deployments frequently fail because we are limited by the lambda execution environment:
- Deployment size should be no more than 500MB because that is the amount of disk space available on the `/tmp` mount of the lambda.
- `sync` duration has to be less than 15 minutes because this is a hard execution timeout for lambda.
See https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
### Use Case
Users use the `s3-deployment` module to deploy static websites to an s3 bucket. On many occasions, the contents of the website directory can be quite big, and contain many files.
See:
- https://github.com/aws/aws-cdk/issues/7571
- https://github.com/aws/aws-cdk/issues/7316
- https://github.com/aws/aws-cdk/issues/6893
### Proposed Solution
No clear solution to this problem yet, this issue is created so we discuss various approaches.
* [ ] :wave: I may be able to implement this feature request
* [ ] :warning: This feature might incur a breaking change
---
This is a :rocket: Feature Request
Contributor guide
Research direction
The issue names the s3-deployment module but no files, tests, or entry points. Start by locating that module and reviewing how deployments use Lambda, then read the linked issues and AWS Lambda limits. Done requires an agreed approach that supports deployments exceeding the 500 MB and 15-minute constraints, but the issue does not yet define the implementation or validation details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100