(ecs): cannot migrate existing ECS service to CODE_DEPLOY deployment controller
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 74
Description
### Describe the bug
Cannot migrate an existing ECS service from ECS deployment controller to CODE_DEPLOY deployment controller using CDK. When attempting to add `deploymentController: { type: DeploymentControllerType.CODE_DEPLOY }` to an existing `ApplicationLoadBalancedFargateService`, the CloudFormation deployment fails with:
Resource handler returned message: "Invalid request provided: Unable to update task definition on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment."
This occurs because CDK changes the task definition format from `{ "Ref": "TaskDefLogicalId" }` to the family name string `"TaskDefFamilyName"` when adding CODE_DEPLOY controller. This format change triggers an ECS UpdateService API call that includes the task definition parameter, which violates the ECS API constraint that CODE_DEPLOY services cannot update task definitions through UpdateService.
**Root Cause**: The `EcsDeploymentGroup` construct validates that `taskDefinition` must be specified as a family name string (line 267-273 in `deployment-group.ts`), but changing from the CloudFormation Ref format to family name format triggers a service update that is rejected by the ECS API.
### Regression Issue
- [x] Select this option if this issue appears to be a regression.
### Last Known Working CDK Library Version
2.50.0
### Expected Behavior
Should be able to seamlessly migrate an existing ECS service from ECS deployment controller to CODE_DEPLOY deployment controller by:
1. Adding `deploymentController: { type: DeploymentControllerType.CODE_DEPLOY }` to the service
2. Creating the `EcsDeploymentGroup` with blue/green configuration
3. Running `cdk deploy`
The deployment should succeed without requiring service recreation.
### Current Behavior
Deployment fails with error:
Resource handler returned message: "Invalid request provided: Unable to update task definition on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment. (Service: Ecs, Status Code
: 400, Request ID: xxx)"
CloudFormation diff shows task definition format changing:
```diff
• taskDefinition: { "Ref": "TaskDefLogicalId" }
+ taskDefinition: "TaskDefFamilyName"
```
This format change triggers UpdateService API call with task definition parameter, which is not allowed for CODE_DEPLOY services per [ECS API documentation](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateService.html).
### Reproduction Steps
### Step 1: Initial Stack (ECS Deployment Controller)
```typescript
import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2';
export class ReproEcsMigrationStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const vpc = new ec2.Vpc(this, 'Vpc', { maxAzs: 2, natGateways: 1 });
const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
const taskDef = new ecs.FargateTaskDefinition(this, 'TaskDef');
taskDef.addContainer('web', {
image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
portMappings: [{ containerPort: 80 }],
});
const alb = new elbv2.ApplicationLoadBalancer(this, 'ALB', { vpc, internetFacing: true });
const listener = alb.addListener('Listener', { port: 80 });
const tg = new elbv2.ApplicationTargetGroup(this, 'TG', {
vpc, port: 80, targetType: elbv2.TargetType.IP,
});
listener.addTargetGroups('TGAttach', { targetGroups: [tg] });
const service = new ecs.FargateService(this, 'Service', {
cluster,
taskDefinition: taskDef,
// deploymentController defaults to ECS
});
service.attachToApplicationTargetGroup(tg);
}
}
```
Deploy: npx cdk deploy
Result: Service uses TaskDefinition: { Ref: "TaskDef54694570" }
### Step 2: Migrate to CODE_DEPLOY (Triggers Bug)
```typescript
// Change the service to use CODE_DEPLOY deployment controller
const service = new ecs.FargateService(this, 'Service', {
cluster,
taskDefinition: taskDef,
deploymentController: { type: ecs.DeploymentControllerType.CODE_DEPLOY }, // ← Add this
});
```
Deploy: npx cdk deploy
Result:
• Task definition format changes to family name: TaskDefinition: "ReproEcsMigrationStackTaskDef1F29B791"
• CloudFormation detects property change and calls UpdateService API
• **Bug:** ECS rejects UpdateService with taskDefinition parameter for CODE_DEPLOY services
### Verified Behavior
Before migration:
```yaml
TaskDefinition:
Ref: TaskDef54694570 # CloudFormation reference
```
After migration:
```yaml
TaskDefinition: ReproEcsMigrationStackTaskDef1F29B791 # Family name string
```
This format change triggers the bug.
### Additional Information/Context
- Related to open issue #25777 (CDK version upgrade causing same error)
- Related to closed issues #23564, #31620, #23370
- Current workaround requires deleting and recreating the service, which is not acceptable for production environments
- This blocks customers from adopting blue/green deployments for existing services
### AWS CDK Library version (aws-cdk-lib)
2.171.0 (also affects 2.50.0 through latest)
### AWS CDK CLI version
2.1029.4 (build 09c0061)
### Node.js Version
v24.9.0
### OS
mac
### Language
TypeScript
### Language Version
_No response_
### Other information
**CloudFormation Template Diff:**
```json
// Before (ECS controller):
"TaskDefinition": {
"Ref": "ServiceTaskDefinition55FA0F15"
}
// After (CODE_DEPLOY controller):
"TaskDefinition": "ServiceTaskDefinitionFamily"
```
**CloudTrail Evidence:**
UpdateService API call includes both `deploymentController` and `taskDefinition` parameters, causing the rejection.
**Workarounds Attempted (all failed):**
1. Using L1 construct override: `cfnService.deploymentController = { type: 'CODE_DEPLOY' }` - Still changes format
2. Explicitly setting family name: `cfnService.taskDefinition = taskDefinition.family` - Causes EcsDeploymentGroup validation error
3. Two-step deployment - First step already fails
**Links:**
- ECS API Constraint: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateService.html
- Related PR #22295: https://github.com/aws/aws-cdk/pull/22295
- Related Issue #25777: https://github.com/aws/aws-cdk/issues/25777
Contributor guide
Research direction
Start by reading deployment-group.ts, especially the taskDefinition validation at lines 267-273, then reproduce the migration with the TypeScript stack and npx cdk deploy. Compare the CloudFormation diff and CloudTrail UpdateService parameters before and after adding CODE_DEPLOY. Done means an existing ECS service can migrate without a rejected UpdateService call containing taskDefinition or requiring recreation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100