cdklabs / cdklabs/cdk-stacksets
StackSetStack file assets fail with S3 NoSuchKey when assets are staged with a custom assetHash (aws-cdk-lib >= 2.254 custom-resource handlers)
- Dominant language
- TypeScript
- Stars
- 118
- Forks
- 25
- Avg merge
- 33m
- Merged PRs (30d)
- 4
Description
## Description
Deploying a `StackSet` whose `StackSetStack` contains an `AwsCustomResource` (or any construct backed by an aws-cdk-lib custom-resource handler) fails on the stack instances with:
```
ResourceLogicalId:AWS679f53fac002430cb0da5b7982bd22872D164C4C, ResourceType:AWS::Lambda::Function,
ResourceStatusReason:Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchKey.
S3 Error Message: The specified key does not exist. (Service: Lambda, Status Code: 400, ...)"
```
This worked with aws-cdk-lib `2.253.1` and breaks with `2.258.1` (the regression window is somewhere in 2.254–2.258). cdk-stacksets version: `0.0.154`.
## Root cause
`StackSetStackSynthesizer.addFileAsset()` derives the template's S3 object key from the *staged asset file name* ([src/stackset-stack.ts](https://github.com/cdklabs/cdk-stacksets/blob/main/src/stackset-stack.ts)):
```ts
const assetFileBaseName = path.basename(asset.fileName);
const s3Filename = assetFileBaseName.split('.')[1] + '.zip';
const objectKey = `${s3Filename}`;
```
But the object that actually lands in the asset bucket is uploaded by the `BucketDeployment` created in the same method:
```ts
new BucketDeployment(parentStack, bucketDeploymentConstructName, {
sources: [Source.asset(assetPath)],
destinationBucket: assetDeployment.assetBucket,
extract: false,
prune: false,
});
```
`Source.asset(assetPath)` re-stages the already-staged asset directory and computes a **fresh content fingerprint** (`FileSystem.fingerprint`). With `extract: false`, the destination object key is that fingerprint-derived key — not the hash embedded in the staged folder name.
These two keys are only identical when the original asset's hash *is* the plain content fingerprint. That assumption broke: since ~aws-cdk-lib 2.254 the generated custom-resource handler singletons stage their code with a **hardcoded custom `assetHash`**, e.g. in `custom-resource-handlers/dist/custom-resources/aws-custom-resource-provider.generated.js`:
```js
code: lambda.Code.fromAsset(path.join(__dirname, "aws-custom-resource-handler"), {
assetHash: "048c51b339711116c3b9feefa3c6aff37c49dba2a4d0f1f531fa8337d9602d82"
}),
```
So for this asset (concrete values from our deployment):
- staged folder: `cdk.out/asset.8b1c21efe0e24332f92ed71e98107d8a27ee1544c62b7b191805e0913fcfdf0f` (derived from the custom `assetHash`)
- template `Code.S3Key`: `8b1c21efe0e24332f92ed71e98107d8a27ee1544c62b7b191805e0913fcfdf0f.zip`
- key actually uploaded by the BucketDeployment: `0528c0095b5b91654052a1e243a4d0125eb054366baf2bf1c791a871d715313a.zip` (= `FileSystem.fingerprint()` of the staged directory)
The stack instance's Lambda then points at a key that never exists in the asset bucket → `NoSuchKey`.
## Reproduction
```ts
const stackSetStack = new StackSetStack(stack, 'StackSetStack', {
assetBuckets: [bucket],
assetBucketPrefix: 'my-assets',
});
new cr.AwsCustomResource(stackSetStack, 'CustomResource', {
onUpdate: {
service: 'GuardDuty',
action: 'listDetectors',
physicalResourceId: cr.PhysicalResourceId.of('id'),
},
policy: cr.AwsCustomResourcePolicy.fromSdkCalls({ resources: ['*'] }),
});
```
Synth with aws-cdk-lib >= 2.258 and compare the stackset template's `Code.S3Key` with the parent stack's `Custom::CDKBucketDeployment` `SourceObjectKeys` — they differ. Deploying fails with `NoSuchKey` on every stack instance.
## Suggested fix
Derive the template key the same way `Source.asset()` will, so they can never diverge:
```ts
const objectKey = `${FileSystem.fingerprint(assetPath)}.zip`;
```
(Equivalently: pass the original asset hash through to `Source.asset(assetPath, { assetHash: ... })` so the upload lands under the key already written to the template — either direction works as long as both sides agree.)
We've verified the `FileSystem.fingerprint` approach fixes the deployment in our landing zone (all asset keys align again, including plain directory assets and pre-zipped bundled assets, which are unaffected no-ops).
## Environment
- cdk-stacksets: 0.0.154
- aws-cdk-lib: 2.258.1 (works on 2.253.1)
- constructs: 10.6.0
- Node.js: 22
Contributor guide
Research direction
Start in src/stackset-stack.ts and inspect StackSetStackSynthesizer.addFileAsset(), then reproduce the issue with an AwsCustomResource and compare the template Code.S3Key with the parent stack's Custom::CDKBucketDeployment SourceObjectKeys. Done means both keys agree for custom-hashed assets and the StackSet instances deploy without S3 NoSuchKey errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100