BucketDeployment: SIGKILL 9 during S3 sync step
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
After a recent CDK update, I started seeing this error on some deploys when using `BucketDeployment`:
> cfn_error: b"Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmps02annce/contents', 's3://beta-us-east-1-main-assets-v1/']' died with ."
The error seems to occur at random, forcing me to frequently retry failed deploys.
I've searched for this same error message but haven't been able to find it. The closest I've found is the issue described at https://github.com/aws/aws-cdk/issues/29862, which is also a SIGKILL but occurs when running the `'aws s3 cp'` command rather than `'aws s3 sync'`. Also, I'm not sure why I'd be running out of memory: The total size of the files I'm deploying is only about 30MB, so I'd expect the 128MB default to be plenty.
I can't say definitively whether this error is caused by a change in CDK or in some other piece of AWS infrastructure.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Last Known Working CDK Version
_No response_
### Expected Behavior
Deploys using `BucketDeployment` should either work or provide a clear error message in the logs when something goes wrong.
### Current Behavior
The error message above is all I see in the logs. It's followed up by a report that says:
> Memory Size: 128 MB Max Memory Used: 128 MB
That suggests that the default memory limit may be the issue… except that I see the same values in the logs when the deploy succeeded. And, again, the files I'm uploading total only 30MB, so what could be using all that memory?
### Reproduction Steps
It's unclear how to reproduce, since the bug is non-deterministic.
### Possible Solution
The improvement proposed at https://github.com/aws/aws-cdk/issues/29898 might address this.
### Additional Information/Context
I'm going to try raising `memoryLimit` to see if it prevents the issue for me. At this point, I don't know.
### CDK CLI Version
2.178.2 (build 89c49cc)
### Framework Version
_No response_
### Node.js Version
20.16.0
### OS
macOS
### Language
Java
### Language Version
Java (17)
### Other information
_No response_
Contributor guide
Research direction
Start at the BucketDeployment entry point and inspect how the S3 sync step is invoked and how memoryLimit is reported. Compare the SIGKILL behavior with issues 29862 and 29898, then try to reproduce it with a deployment of similar size. Done means identifying the cause and either fixing the failure or providing a clear diagnostic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100