aws / aws/aws-cdk

cdk.aws_s3_deployment.BucketDeployment: Cross-stack sharing of BucketDeployment constructs causes "Template Format" errors

Open
#33,803 2 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-s3-deployment bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

Cross-stack sharing of BucketDeployment constructs causes `Template format error: Every Value member must be a string.` This appears to be related to the manner in which the `SourceObjectKeys` attribute is exported once the Stack is built, but the error is vague and it is not immediately obvious as to why this behaviour interferes with deployment.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Last Known Working CDK Version

_No response_

### Expected Behavior

While I understand that exporting the entire BucketDeployment construct may not necessarily be the best option for cross-stack sharing, I would expect (at minimum) that exporting the result of `Fn.select(0, .objectKeys)` would not trigger a `Template format error: Every Value member must be a string.` It would also be good to improve the error related to this case where a user attempts to share a list of of SourceObjectkeys with something such as: `Template format error: Exporting an array of SourceObjectkeys is not supported. Use Fn.select() instead.`

### Current Behavior

Consider the following Stack as an example which exports multiple BucketDeployment constructs:

```
import * as cdk from 'aws-cdk-lib';
import * as s3 from 'aws-cdk-lib/aws-s3';
import { Construct } from 'constructs';
import { NetworkStack } from './network-stack';
import path = require('path');

export interface PcScriptsStackProps extends cdk.StackProps {
networkStack: NetworkStack;
}

export class PcScriptsStack extends cdk.Stack {
// Public readonly properties
public readonly scriptBucket: s3.Bucket;
public readonly onHeadNodeConfiguredScript: cdk.aws_s3_deployment.BucketDeployment;
public readonly onFleetNodeConfiguredScript: cdk.aws_s3_deployment.BucketDeployment;

constructor(scope: Construct, id: string, props: PcScriptsStackProps) {
super(scope, id, props);

// Create an S3 bucket for hosting our scripts
this.scriptBucket = new s3.Bucket(this, 'scriptbucket', {
removalPolicy: cdk.RemovalPolicy.DESTROY,
autoDeleteObjects: true,
encryption: s3.BucketEncryption.S3_MANAGED,
enforceSSL: true,
blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
});

// Create a BucketDeployment to upload assets/actions/ to scriptBucket with a prefix of actions/...
this.onFleetNodeConfiguredScript = new cdk.aws_s3_deployment.BucketDeployment(this, 'FleetNodeOnConfigured', {
sources: [cdk.aws_s3_deployment.Source.asset(path.join(__dirname, '..', 'assets', 'actions', 'fleet'))],
destinationBucket: this.scriptBucket,
destinationKeyPrefix: "actions/fleet/",
retainOnDelete: false
});

this.onHeadNodeConfiguredScript = new cdk.aws_s3_deployment.BucketDeployment(this, 'HeadNodeOnConfigured', {
sources: [cdk.aws_s3_deployment.Source.asset(path.join(__dirname, '..', 'assets', 'actions', 'head'))],
destinationBucket: this.scriptBucket,
destinationKeyPrefix: "actions/head/",
retainOnDelete: false
});
}
}
```

The stack will pass a `cdk synth`, but will fail once a deployment is attempted. Even though all constructs will be deployed successfully, the stack will still fail with `Template format error: Every Value member must be a string.` When reviewing the synthesised stack in the CloudFormation console, we can observe that array exports are being created based on the `SourceObjectKeys` attribute of the BucketDeployment construct. See excerpts below:

` "ExportsOutputFnGetAttHeadNodeOnConfiguredCustomResource0FC96C29SourceObjectKeysFFA1E1D5": { "Value": { "Fn::GetAtt": [ "HeadNodeOnConfiguredCustomResource0FC96C29", "SourceObjectKeys" ] }, "Export": { "Name": "PcScriptsStack:ExportsOutputFnGetAttHeadNodeOnConfiguredCustomResource0FC96C29SourceObjectKeysFFA1E1D5" } },`

`"Fn::GetAtt": [ "HeadNodeOnConfiguredCustomResource0FC96C29", "SourceObjectKeys" ]`

...in which `"SourceObjectKeys"` resolves to an array:

`"SourceObjectKeys": [ "63c0857f21787e8158b5d5afafe6da1adc20f0060c24aeae1c6477a9a4d49917.zip" ],`

Since CFN expects exports to be simple key-value pairs, we encounter the Template format error mentioned above. This happens even if `Fn.select(0, onFleetNodeConfiguredScript.objectKeys)` were used to extract a single object key for export rather than the entire construct. Template attached for reference.

[script_generated_template.json](https://github.com/user-attachments/files/19291376/script_generated_template.json)

### Reproduction Steps

Thanks to @mrmilosz, here's a snippet that reproduces the issue.

```
import * as cdk from 'aws-cdk-lib/core';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as s3_deployment from 'aws-cdk-lib/aws-s3-deployment';

const app = new cdk.App();

const env: cdk.Environment = {
account: '0123456789012',
region: 'us-east-1',
};

const stack = new cdk.Stack(app, 'Stack', {
env,
});

const destinationBucket = new s3.Bucket(stack, 'Bucket');

const deployment = new s3_deployment.BucketDeployment(stack, 'Deployment', {
sources: [s3_deployment.Source.data('hello.txt', 'Hello, world!')],
destinationBucket,
});

const anotherStack = new cdk.Stack(app, 'AnotherStack', {
env,
});

new lambda.Function(anotherStack, 'ObjectKeyLoggingFunction', {
code: lambda.Code.fromInline('exports.handler = (event) => { console.log(process.env.OBJECT_KEY); };'),
handler: 'index.handler',
runtime: lambda.Runtime.NODEJS_LATEST,
environment: {
OBJECT_KEY: cdk.Fn.select(0, deployment.objectKeys),
},
});

```

### Possible Solution

_No response_

### Additional Information/Context

My use case here involved building an AWS ParallelCluster deployment. Custom Action hooks are exposed by the configuration, and I was using the BucketDeployment construct to upload action scripts before extracting their keys from the destination bucket and populating appropriate URIs in the config stack.

### CDK CLI Version

2.1003.0 (build b242c23)

### Framework Version

_No response_

### Node.js Version

23.7.0

### OS

MacOS 15.3

### Language

TypeScript

### Language Version

TypeScript (5.8.2)

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the cross-stack case with the BucketDeployment and Fn.select(0, deployment.objectKeys) examples, then inspect the synthesized template and the SourceObjectKeys export shown in script_generated_template.json. Done means the cross-stack reference no longer produces an invalid CloudFormation export, including when selecting one object key, and the resulting deployment template is accepted.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.