aws / aws/aws-cdk

(ecs-patterns): Creating a NetworkLoadBalancedFargateService with an imported NetworkLoadBalancer creates the Listener in the stack the NLB was looked up in, creating a circular dependency

Open
#26,806 3 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-ecs-patterns bug documentation effort/small p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
71

Description

### Describe the bug

Hi all,

I created a NetworkLoadBalancedFargateService using a NLB that I looked up by tag, as it is associated with a number of other services at my organization. The service is created as part of an `EcsStack` and the lookup happens in a `LookupStack`.

This all went fine and got deployed into all of our various environments. However, when trying to make a change to the resources provisioned, the `EcsStack` and the `LookupStack` have become circularly dependent due to the fact that, for some reason, the `Listener` and the `TargetGroup` created by the `EcsStack` is actually part of the `LookupStack`'s CloudFormation template:

```
# in lookup-stack.template.json
"...": {
"Type": "AWS::ElasticLoadBalancingV2::Listener",
"Properties": {
"DefaultActions": [
{
"TargetGroupArn": {
"Ref": "..."
},
"Type": "forward"
}
],
"LoadBalancerArn": "...",
"Port": ..,
"Protocol": "TCP"
},
"Metadata": {
"aws:cdk:path": "..."
}
},
"...": {
"Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
"Properties": {
"Port": ...,
"Protocol": "TCP",
"TargetType": "ip",
"VpcId": "..."
},
"Metadata": {
"aws:cdk:path": "..."
}
```
This is surprising, and when I was running some tests on how to break this circular dependency, I noticed that creating a Listener as an L2 construct in the `EcsStack` properly associates it with the `EcsStack` CloudFormation template rather than the `LookupStack` template. I'm also now at a loss for how to break this dependency - especially given that this service is deployed into many accounts, so ideally it would be fixable in a single CDK deployment.

### Expected Behavior

My expectation is that Resources created in a stack are associated with the CloudFormation template of THAT stack, not any other stack. Especially not one that immediately creates a circular dependency.

### Current Behavior

The Listener and Target group created by the NetworkLoadBalancedFargateService L3 construct are part of the stack that looked up the NLB, rather than the stack creating the NetworkLoadBalancedFargateService construct.

I then tried to just move the lookup of the NLB into the `LookupStack`, but I get the circular dependency error:

```
Export {account-name}stack:ExportsOutputRefdev...PublicListenerECSGroup1D50C3306D93B20F cannot be deleted as it is in use by {account-name}-ecs-stack
```
### Reproduction Steps

Create a NetworkLoadBalancer outside of CDK that has some identifiable tag.

Make two stacks:

Stack 1:
Look up NLB by tag.
Return the NLB.

Stack 2:
Take the NLB returned by Stack 1 as an argument.
Create a NetworkLoadBalancedFargateService with the above NLB as the NLB.

The listener and target group of Stack 2's service will be part of Stack 1.

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.92.0

### Framework Version

_No response_

### Node.js Version

v20.2

### OS

Ubuntu

### Language

Go

### Language Version

1.21.0

### Other information

I can see what I should have done in the first place, but now I'm more interested in figuring out how I can unwind these mutually-dependent stacks without service disruption in production.

Contributor guide

Open the contributing guide

Research direction

Start at the NetworkLoadBalancedFargateService construct and its handling of an imported NetworkLoadBalancer, then reproduce the two-stack lookup and service setup described in the issue. Verify the synthesized templates and deployment behavior; done means the listener and target group stay with the service stack without creating a circular dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.