cdk.aws_s3_deployment.BucketDeployment: Cross-stack sharing of BucketDeployment constructs causes "Template Format" errors
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
Cross-stack sharing of BucketDeployment constructs causes `Template format error: Every Value member must be a string.` This appears to be related to the manner in which the `SourceObjectKeys` attribute is exported once the Stack is built, but the error is vague and it is not immediately obvious as to why this behaviour interferes with deployment.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Last Known Working CDK Version
_No response_
### Expected Behavior
While I understand that exporting the entire BucketDeployment construct may not necessarily be the best option for cross-stack sharing, I would expect (at minimum) that exporting the result of `Fn.select(0, .objectKeys)` would not trigger a `Template format error: Every Value member must be a string.` It would also be good to improve the error related to this case where a user attempts to share a list of of SourceObjectkeys with something such as: `Template format error: Exporting an array of SourceObjectkeys is not supported. Use Fn.select() instead.`
### Current Behavior
Consider the following Stack as an example which exports multiple BucketDeployment constructs:
```
import * as cdk from 'aws-cdk-lib';
import * as s3 from 'aws-cdk-lib/aws-s3';
import { Construct } from 'constructs';
import { NetworkStack } from './network-stack';
import path = require('path');
export interface PcScriptsStackProps extends cdk.StackProps {
networkStack: NetworkStack;
}
export class PcScriptsStack extends cdk.Stack {
// Public readonly properties
public readonly scriptBucket: s3.Bucket;
public readonly onHeadNodeConfiguredScript: cdk.aws_s3_deployment.BucketDeployment;
public readonly onFleetNodeConfiguredScript: cdk.aws_s3_deployment.BucketDeployment;
constructor(scope: Construct, id: string, props: PcScriptsStackProps) {
super(scope, id, props);
// Create an S3 bucket for hosting our scripts
this.scriptBucket = new s3.Bucket(this, 'scriptbucket', {
removalPolicy: cdk.RemovalPolicy.DESTROY,
autoDeleteObjects: true,
encryption: s3.BucketEncryption.S3_MANAGED,
enforceSSL: true,
blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
});
// Create a BucketDeployment to upload assets/actions/ to scriptBucket with a prefix of actions/...
this.onFleetNodeConfiguredScript = new cdk.aws_s3_deployment.BucketDeployment(this, 'FleetNodeOnConfigured', {
sources: [cdk.aws_s3_deployment.Source.asset(path.join(__dirname, '..', 'assets', 'actions', 'fleet'))],
destinationBucket: this.scriptBucket,
destinationKeyPrefix: "actions/fleet/",
retainOnDelete: false
});
this.onHeadNodeConfiguredScript = new cdk.aws_s3_deployment.BucketDeployment(this, 'HeadNodeOnConfigured', {
sources: [cdk.aws_s3_deployment.Source.asset(path.join(__dirname, '..', 'assets', 'actions', 'head'))],
destinationBucket: this.scriptBucket,
destinationKeyPrefix: "actions/head/",
retainOnDelete: false
});
}
}
```
The stack will pass a `cdk synth`, but will fail once a deployment is attempted. Even though all constructs will be deployed successfully, the stack will still fail with `Template format error: Every Value member must be a string.` When reviewing the synthesised stack in the CloudFormation console, we can observe that array exports are being created based on the `SourceObjectKeys` attribute of the BucketDeployment construct. See excerpts below:
` "ExportsOutputFnGetAttHeadNodeOnConfiguredCustomResource0FC96C29SourceObjectKeysFFA1E1D5": { "Value": { "Fn::GetAtt": [ "HeadNodeOnConfiguredCustomResource0FC96C29", "SourceObjectKeys" ] }, "Export": { "Name": "PcScriptsStack:ExportsOutputFnGetAttHeadNodeOnConfiguredCustomResource0FC96C29SourceObjectKeysFFA1E1D5" } },`
`"Fn::GetAtt": [ "HeadNodeOnConfiguredCustomResource0FC96C29", "SourceObjectKeys" ]`
...in which `"SourceObjectKeys"` resolves to an array:
`"SourceObjectKeys": [ "63c0857f21787e8158b5d5afafe6da1adc20f0060c24aeae1c6477a9a4d49917.zip" ],`
Since CFN expects exports to be simple key-value pairs, we encounter the Template format error mentioned above. This happens even if `Fn.select(0, onFleetNodeConfiguredScript.objectKeys)` were used to extract a single object key for export rather than the entire construct. Template attached for reference.
[script_generated_template.json](https://github.com/user-attachments/files/19291376/script_generated_template.json)
### Reproduction Steps
Thanks to @mrmilosz, here's a snippet that reproduces the issue.
```
import * as cdk from 'aws-cdk-lib/core';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as s3_deployment from 'aws-cdk-lib/aws-s3-deployment';
const app = new cdk.App();
const env: cdk.Environment = {
account: '0123456789012',
region: 'us-east-1',
};
const stack = new cdk.Stack(app, 'Stack', {
env,
});
const destinationBucket = new s3.Bucket(stack, 'Bucket');
const deployment = new s3_deployment.BucketDeployment(stack, 'Deployment', {
sources: [s3_deployment.Source.data('hello.txt', 'Hello, world!')],
destinationBucket,
});
const anotherStack = new cdk.Stack(app, 'AnotherStack', {
env,
});
new lambda.Function(anotherStack, 'ObjectKeyLoggingFunction', {
code: lambda.Code.fromInline('exports.handler = (event) => { console.log(process.env.OBJECT_KEY); };'),
handler: 'index.handler',
runtime: lambda.Runtime.NODEJS_LATEST,
environment: {
OBJECT_KEY: cdk.Fn.select(0, deployment.objectKeys),
},
});
```
### Possible Solution
_No response_
### Additional Information/Context
My use case here involved building an AWS ParallelCluster deployment. Custom Action hooks are exposed by the configuration, and I was using the BucketDeployment construct to upload action scripts before extracting their keys from the destination bucket and populating appropriate URIs in the config stack.
### CDK CLI Version
2.1003.0 (build b242c23)
### Framework Version
_No response_
### Node.js Version
23.7.0
### OS
MacOS 15.3
### Language
TypeScript
### Language Version
TypeScript (5.8.2)
### Other information
_No response_
Contributor guide
Research direction
Start by reproducing the cross-stack case with the BucketDeployment and Fn.select(0, deployment.objectKeys) examples, then inspect the synthesized template and the SourceObjectKeys export shown in script_generated_template.json. Done means the cross-stack reference no longer produces an invalid CloudFormation export, including when selecting one object key, and the resulting deployment template is accepted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100