aws / aws/aws-cdk

app-staging-synthesizer: Deletion of DefaultStagingStack fails when it does not contains any s3 asset

Open
#27,834 0 comments 2 reactions 0 assignees View on GitHub
@aws-cdk/app-staging-synthesizer-alpha bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

When using ` AppStagingSynthesizer.defaultResources` and we don't use any asset in stacks in the app, CFn `DependsOn` properties are not rendered properly on `DefaultStagingStack`.

This at least makes deleting the staging stack fail because a bucket policy sometimes gets deleted before `autoDeleteObject` custom resource ignoring its dependency.

### Expected Behavior

`DependsOn` properties are rendered.

### Current Behavior

`DependsOn` properties are not rendered.

### Reproduction Steps

Use the following code for cdk app.

```ts
// bin/cdk.ts
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { AppStagingSynthesizer, DeploymentIdentities } from '@aws-cdk/app-staging-synthesizer-alpha';

const app = new cdk.App({
defaultStackSynthesizer: AppStagingSynthesizer.defaultResources({
appId: 'my-app-id',
deploymentIdentities: DeploymentIdentities.defaultBootstrapRoles({ bootstrapRegion: 'us-east-1' }),
}),
});

// stack without any asset
new cdk.Stack(app, 'StagingSynthesizerStack');
```

Run `cdk synth` and check the output template (`StagingStack-my-app-id2-ACCOUNT-REGION.template.json`).

You can find this resource:

```json
"CdkStagingBucketAutoDeleteObjectsCustomResource800E998D": {
"Type": "Custom::S3AutoDeleteObjects",
"Properties": {
"ServiceToken": {
"Fn::GetAtt": [
"CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F",
"Arn"
]
},
"BucketName": {
"Ref": "CdkStagingBucket1636058C"
}
},
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete"
},
```

Where it should be:

```diff
"CdkStagingBucketAutoDeleteObjectsCustomResource800E998D": {
"Type": "Custom::S3AutoDeleteObjects",
"Properties": {
"ServiceToken": {
"Fn::GetAtt": [
"CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F",
"Arn"
]
},
"BucketName": {
"Ref": "CdkStagingBucket1636058C"
}
},
+ "DependsOn": [
+ "CdkStagingBucketPolicy42BD1F92"
+ ],
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete"
},
```

### Possible Solution
Here's my observation:

The root cause for this issue is that we add resources to an CDK app AFTER cdk runs `prepareApp` function.

https://github.com/aws/aws-cdk/blob/fce26b635d0ef2c443ca6d4e390a90f586130f05/packages/aws-cdk-lib/core/lib/private/synthesis.ts#L45

`Bucket` resource can be added even after `prepareApp` here (it's added on the first time `addFileAsset` is called) : https://github.com/aws/aws-cdk/blob/fce26b635d0ef2c443ca6d4e390a90f586130f05/packages/%40aws-cdk/app-staging-synthesizer-alpha/lib/app-staging-synthesizer.ts#L320

When we use any (s3) assets in the app (e.g. Lambda code asset), `addFileAsset` is called before `prepareApp`, so in that case it works as expected.

In `prepareApp` function, we search for all the dependencies between nodes and register them as a CFn resource dependencies. That's why dependencies added after `prepareApp` are not rendered at all.

https://github.com/aws/aws-cdk/blob/fce26b635d0ef2c443ca6d4e390a90f586130f05/packages/aws-cdk-lib/core/lib/private/prepare-app.ts#L17-L28

So the possible solution can be to always create a staging bucket first to avoid adding it after `prepareApp`. To do that, for example, we can add `this.getCreateBucket();` to the stack constructor here:

https://github.com/aws/aws-cdk/blob/fce26b635d0ef2c443ca6d4e390a90f586130f05/packages/%40aws-cdk/app-staging-synthesizer-alpha/lib/default-staging-stack.ts#L241-L246

I'm not quite sure if there is more thorough solution (given that there are possibly other resources other than a staging bucket that can be added after `prepareApp`). Afaik it seems only a staging bucket is causing this issue because the bucket is used to upload CFn templates in the app (that's why it can be added after `prepareApp`).

### Additional Information/Context

On the other hand, `DependsOn` of `CustomResourceProviderHandler` is rendered properly.

```json
"CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F": {
"Type": "AWS::Lambda::Function",
"Properties": {
},
"DependsOn": [
"CustomS3AutoDeleteObjectsCustomResourceProviderRole3B1BD092"
]
}
```

This is because custom resource provider uses `CfnResource.addDependency` directly instead of `Node.addDependency`.

https://github.com/aws/aws-cdk/blob/7c497ee998976b69ad1e2cd3e68f30cb593f8ded/packages/aws-cdk-lib/core/lib/custom-resource-provider/custom-resource-provider.ts#L313

### CDK CLI Version

2.104.0

### Framework Version

2.104.0

### Node.js Version

18

### OS

macOS

### Language

TypeScript

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the provided TypeScript app and inspect the synthesized StagingStack template. Start with default-staging-stack.ts around the constructor and getCreateBucket(), then trace addFileAsset() in app-staging-synthesizer.ts alongside prepare-app.ts and synthesis.ts. Done means the staging bucket auto-delete resource retains its DependsOn relationship when no application asset is used, with deletion behavior covered by regression testing.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.