aws / aws/aws-cdk

core: referencing arrays/lists when using `crossRegionReferences` fails with "SerializationException: Start of list found where not expected"

Open
#31,655 1 comment 0 reactions 0 assignees View on GitHub
@aws-cdk/core bug effort/medium p1
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
71

Description

### Describe the bug

When creating stacks with `crossRegionReferences` enabled, and the stack exposes a list/array value that is to be read in another region, the `Custom::CrossRegionExportWriter` resource will fail during creation with the error "SerializationException: Start of list found where not expected". Reading string values works fine, but if it's an array/list, it will fail.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Last Known Working CDK Version

_No response_

### Expected Behavior

I expect to be able to reference lists/arrays from a stack in another region like I would strings.

### Current Behavior

The `Custom::CrossRegionExportWriter` resource fails during creation with the error "SerializationException: Start of list found where not expected":

```
BugVPCStack-us-west-2
BugVPCStack-us-west-2: deploying... [1/4]
Bug-VPC1-Requests-VPC2: start: Building bc980b3b9776ac14eeed5eeecde033423deae6a101589e745c39d5e53e119d87:current_account-us-west-2
Bug-VPC1-Requests-VPC2: success: Built bc980b3b9776ac14eeed5eeecde033423deae6a101589e745c39d5e53e119d87:current_account-us-west-2
Bug-VPC2-Accepts-VPC1: start: Building a7bb53c2fb4a92a7dcc01a0863f4145f4161926cdb3d34becc9fe4a65ce54566:current_account-us-east-1
Bug-VPC2-Accepts-VPC1: success: Built a7bb53c2fb4a92a7dcc01a0863f4145f4161926cdb3d34becc9fe4a65ce54566:current_account-us-east-1
BugVPCStack-us-west-2: creating CloudFormation changeset...
3:14:45 PM | UPDATE_FAILED | Custom::CrossRegionExportWriter | ExportsWriteruseast10F67B507DDE2E818
Received response status [FAILED] from custom resource. Message returned: SerializationException: Start of list found where not expected
at throwDefaultError (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:840:20)
at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:849:5
at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-ssm/dist-cjs/index.js:7021:14)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
at async Promise.all (index 0)
at async u (/var/task/index.js:3:330) (RequestId: f443b2b6-c5f3-49e6-98ce-eec9f39ef19d)

❌ BugVPCStack-us-west-2 failed: Error: The stack named BugVPCStack-us-west-2 failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: SerializationException: Start of list found where not expected
at throwDefaultError (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:840:20)
at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:849:5
at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-ssm/dist-cjs/index.js:7021:14)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
at async Promise.all (index 0)
at async u (/var/task/index.js:3:330) (RequestId: f443b2b6-c5f3-49e6-98ce-eec9f39ef19d)
at FullCloudFormationDeployment.monitorDeployment (/Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:458:10567)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:461:200334)
at async /Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:461:181756

❌ Deployment failed: Error: The stack named BugVPCStack-us-west-2 failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: SerializationException: Start of list found where not expected
at throwDefaultError (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:840:20)
at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:849:5
at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-ssm/dist-cjs/index.js:7021:14)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
at async Promise.all (index 0)
at async u (/var/task/index.js:3:330) (RequestId: f443b2b6-c5f3-49e6-98ce-eec9f39ef19d)
at FullCloudFormationDeployment.monitorDeployment (/Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:458:10567)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:461:200334)
at async /Users/aang/Code/Work/repos/core/cdk/Vpc-Peering/node_modules/aws-cdk/lib/index.js:461:181756

The stack named BugVPCStack-us-west-2 failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: SerializationException: Start of list found where not expected
at throwDefaultError (/var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:840:20)
at /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/smithy-client/dist-cjs/index.js:849:5
at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-ssm/dist-cjs/index.js:7021:14)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22
at async Promise.all (index 0)
at async u (/var/task/index.js:3:330) (RequestId: f443b2b6-c5f3-49e6-98ce-eec9f39ef19d)
```

### Reproduction Steps

By default, the app will be created in a way that does not trigger the bug. To deploy the app as-is:
```
cdk bootstrap
cdk deploy --all --app 'node app.ts'
```

This should create the stacks successfully. Then run another deployment with a context variable:
```
cdk deploy --all --app 'node app.ts' --context readIPv6=true
```

This will instruct the app to reference list values in a cross-region manager and trigger the bug.

The bug occurs whether you create the stack with or without the context variable set; that is to say, if you create the stack with the context variable, the bug will also be triggered in that circumstance.

**app.ts:**
```typescript
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';

interface VpcStackProps extends cdk.StackProps {
readonly cidr: string;
}

class VpcStack extends cdk.Stack {
public readonly vpcId: string;
public readonly vpcCidrIpV4: string;
public readonly vpcCidrIpV6: string;
public readonly vpcCidrIpV6List: string[];

public constructor(scope: cdk.App, id: string, props: VpcStackProps) {
super(scope, id, props);

const vpc = new ec2.Vpc(this, 'Bug VPC', {
ipAddresses: ec2.IpAddresses.cidr(props.cidr),
ipProtocol: ec2.IpProtocol.DUAL_STACK,
createInternetGateway: false,
maxAzs: 3,
subnetConfiguration: [
{
cidrMask: 28,
name: 'private',
subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
}
]
})

this.vpcId = vpc.vpcId
this.vpcCidrIpV4 = vpc.vpcCidrBlock

// Both the selected value and the original list value will trigger the bug
this.vpcCidrIpV6 = cdk.Fn.select(0, vpc.vpcIpv6CidrBlocks)
this.vpcCidrIpV6List = vpc.vpcIpv6CidrBlocks
}
}

function createVpcStack(region: string, cidr: string) {
const stack = new VpcStack(app, `BugVPCStack-${region}`, {
env: { region },
crossRegionReferences: true,
cidr
})

return stack
}

const ec2 = cdk.aws_ec2
const app = new cdk.App();

const vpc1Stack = createVpcStack('us-west-2', '10.16.0.0/16')
const vpc2Stack = createVpcStack('us-east-1', '10.17.0.0/16')

const requesterStack = new cdk.Stack(app, `Bug-VPC1-Requests-VPC2`, {
env: { region: vpc1Stack.region },
crossRegionReferences: true
})

const readIPv6 = app.node.tryGetContext('readIPv6')
const values = [
vpc1Stack.vpcId!,
vpc1Stack.vpcCidrIpV4!,
vpc2Stack.vpcId!,
vpc2Stack.vpcCidrIpV4!,
...(readIPv6
? [
vpc1Stack.vpcCidrIpV6!,
vpc2Stack.vpcCidrIpV6!,
]
: [])
]

new cdk.aws_ssm.StringListParameter(requesterStack, 'sourceVpc', {
stringListValue: values,
})

if (readIPv6) new cdk.aws_ssm.StringListParameter(requesterStack, 'sourceVpcIpv6', {
stringListValue: vpc2Stack.vpcCidrIpV6List!,
})

const accepterStack = new cdk.Stack(app, `Bug-VPC2-Accepts-VPC1`, {
env: { region: vpc2Stack.region },
crossRegionReferences: true
})

new cdk.aws_ssm.StringListParameter(accepterStack, 'sourceVpc', {
stringListValue: values,
})

if (readIPv6) new cdk.aws_ssm.StringListParameter(accepterStack, 'sourceVpcIpv6', {
stringListValue: vpc1Stack.vpcCidrIpV6List!,
})
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.158.0

### Framework Version

_No response_

### Node.js Version

20.15.0

### OS

macOS 14.6.1

### Language

TypeScript

### Language Version

5.5.4

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the reproduction in app.ts and run cdk bootstrap followed by the two cdk deploy commands, including --context readIPv6=true. Trace the Custom::CrossRegionExportWriter failure when vpcCidrIpV6List is referenced across regions; done means cross-region list or array values deploy successfully without the SerializationException.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.