aws / aws/containers-roadmap

[ECS][request]: First-class service-level image updates without task definition re-registration

Open
#2,680 1 comment 12 reactions 1 assignee Claimed by @nikitadahiya View on GitHub
ECS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

## General Ask

Allow updating container images for an ECS service as a first-class operation, without creating a new task definition revision. This removes drift/races with other controllers that manage task definition shape (CDK/CloudFormation, Terraform), improves deployment speed, reduces API pressure from ECS control plane, and should help simplify observability and rollbacks as well.

## Understanding of the problem

- Today, changing container images requires re-registering a new task definition revision. If any other controller (e.g., CDK/CloudFormation) also modifies task definitions, concurrent or interleaved updates cause “eternal drift” and race conditions. Teams must either:
- Choose a single controller (often impractical when infra is CFN-managed but app deployments are frequent), or
- Implement fragile coordination to serialize deployments, which slows delivery and still occasionally races.
- Additionally, it’s hard to derive a single, strongly consistent “desired” definition in real time; CloudFormation artifacts, change sets, and ECS Describe APIs don’t expose a fully resolved, final artifact for deploy tooling to safely mutate.

## Current behavior and limitations

- In order to perform a deploy today, it requires `RegisterTaskDefinition` + `UpdateService` with an optional `DescribeTaskDefinition` to point service(s) to the new revision.
- Any shape change by CFN/CDK can be lost if another system re-registers from an older base.
- “forceNewDeployment” cannot change image; it only rolls tasks on the existing task definition.
- CloudFormation/ChangeSets/SSM indirections are often too slow or prone to service limits for routine image bumps, and still don’t eliminate drift when mixed with ECS-API deploys.

The major impact and limiting factor from this comes from drift and inconsistent environments across services/clusters, API rate limits on tens-to-hundreds of ECS clusters getting a deploy at once.

The current solution is to request to service quota bumps and slow down deploys or "coordinate" deploys between different controllers, all leading to sub-optimal solutions.

## Proposal
Any solution to solve this problem place would be great. One thing came to mind is:

- Introduce a service-scoped, first-class image override that is applied at deployment time without creating a new task definition revision.
- Treat the task definition as the “shape” and the service as the place where image versions are declared and rolled out.
- Pin to digest at rollout time for auditability and reproducibility (optional)
- API sketch
- ECS: new operation(s)
- `PutServiceImageOverrides` to set per-container `image` or `imageDigest`.
- `GetServiceImageOverrides` to retrieve current overrides and pinned digests.
- `UpdateServiceDeployment` optionally referencing an “image override set” and triggering a rollout.
- Request fields
- `service`, `cluster`, `clientToken`
- `containerImageOverrides`: array of { containerName, image (tag or digest), resolveTagToDigest: boolean }
- `expectedTaskDefinitionArn` (optimistic concurrency guard against shape changes)
- Responses/events
- Return resolved digests for each container.
- Emit EventBridge events for “image override set”, “digest resolved”, “deployment started/completed”.
- IAM
- Fine-grained actions: `ecs:PutServiceImageOverrides`, `ecs:GetServiceImageOverrides`, distinct from `ecs:RegisterTaskDefinition`.
- CloudFormation/CDK
- New `AWS::ECS::Service` property: `ImageOverrides` (map container → image/tag/digest).
- Drift detection shows image overrides distinctly from task definition changes.
- CLI examples
```bash
aws ecs put-service-image-overrides \
--cluster prod \
--service web \
--container-image-overrides '[{"containerName":"app","image":"123456789012.dkr.ecr.us-east-1.amazonaws.com/app:web-2025-09-26"}]' \
--expected-task-definition-arn arn:aws:ecs:us-east-1:123456789012:task-definition/web:57
--use-latest-task-definition # default
```
```bash
aws ecs describe-services --cluster prod --services web \
--query 'services[0].imageOverrides'
```
### Caution and backwards compatibility

Ofcourse this is a massive change to how things work today, so its merely a proposal/first-draft idea to get some conversations going. There are other alternatives like polling against some "git source" or moving to Git Ops like workloads for ECS on EC2 as well, which would be nice.

Regarding backwards compatibility, something like the above would be an opt-in for clusters and not impact existing task definition based service rollouts.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.