aws / aws/aws-cdk

@aws-cdk/aws-s3-deployment: BucketDeployment fails to remove custom resource

Open
#23,708 4 comments 4 reactions 0 assignees View on GitHub
@aws-cdk/aws-s3-deployment bug p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

When a BucketDeployment is removed (or renamed), the associated custom resource lambda will fail due to permission errors related to the "cloudfront:CreateInvalidation" action.

My running theory for this is because bucket invalidation occurs regardless of whether the BucketDeployment resource is being added/modified or deleted (see https://github.com/aws/aws-cdk/blob/8a7ec37e0085e952b59bebab1ecc880167a0691f/packages/%40aws-cdk/aws-s3-deployment/lib/lambda/index.py#L119)

This makes sense, as you might want to make sure that the files associated with the deleted BucketDeployment are not sticking around in the cache. However, I think that the permissions that BucketDeployment adds to allow the invalidation (https://github.com/aws/aws-cdk/blob/8a7ec37e0085e952b59bebab1ecc880167a0691f/packages/%40aws-cdk/aws-s3-deployment/lib/bucket-deployment.ts#L341) have already been removed at this point so the invalidation fails.

### Expected Behavior

The BucketDeployment resource to delete cleanly.

### Current Behavior

The custom resource delete fails. It doesn't fail the whole stack, it seems to just ignore the error.

Next deployment does not show the same error, so it is possible that resources are in an inconsistent state.

### Reproduction Steps

- Deploy a CDK stack with a BucketDeployment.
- Rename or delete that BucketDeployment in a subsequent deploy.
- Check the CloudFormation stack events

### Possible Solution

I'll have to have a think. The easiest fix would be to just not invalidate the distribution on delete. However, this might leave resources around in the cache that people don't know about.

I'm not sure how the order could be changed so that the custom resource lambda fires before the permissions are removed.

### Additional Information/Context

_No response_

### CDK CLI Version

2.51.1 (build 3d30cdb)

### Framework Version

_No response_

### Node.js Version

16.18.1

### OS

MacOS

### Language

Typescript

### Language Version

4.9.3

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with packages/@aws-cdk/aws-s3-deployment/lib/bucket-deployment.ts around the permission setup and packages/@aws-cdk/aws-s3-deployment/lib/lambda/index.py around invalidation handling. Reproduce the issue by removing or renaming a BucketDeployment, then inspect the CloudFormation stack events; done means deletion completes without the custom resource permission error while preserving the intended cache behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.