aws / aws/containers-roadmap

[ECS] [request]: Expose service desired-state revisions to support GitOps and external controllers

Open
#2,854 0 comments 0 reactions 0 assignees View on GitHub
ECS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

ECS currently provides lifecycle events through EventBridge (ECS Deployment State Change, ECS Task State Change, etc.), which are useful for reacting to runtime transitions.

What is currently missing is a lightweight, consistent way for external control loops to observe changes to an **ECS service's desired-state configuration**.

I'm building an open-source GitOps reconciliation controller for ECS and would like ECS to expose a primitive that allows external controllers to efficiently determine whether a service's desired configuration has changed without repeatedly fetching and diffing the complete service configuration.
I operate environments with 30+ ECS services in a cluster, where I want GitOps workflows that behave like a proper continuous reconciliation system rather than relying on CI/CD push-based workarounds.

The ideal solution could be one or more of the following:

1. A monotonic `desiredStateRevision` identifier on the Service, returned by `DescribeServices`, which increments whenever the service's desired configuration changes. *(Note: If strict monotonic integers are too complex for the distributed backend, a deterministic state hash or ETag would also work).*
2. A resumable change stream using a sequence token / resourceVersion-style mechanism, allowing an external controller to resume observation after being offline without missing changes.
3. EventBridge events for service desired-configuration changes, in addition to the existing deployment and task lifecycle events.

A revision or ETag would allow controllers to determine whether a full configuration read and reconciliation are necessary, while avoiding continuous full-state polling.

A push-based workflow can deploy a change when Git changes, but it does not provide a reliable control loop for continuously determining whether the actual ECS state still matches the desired state.

For example, infrastructure changes can indirectly modify services, operators can make manual changes, Terraform can register task-definition revisions, and other AWS APIs or automation can modify service configuration outside the GitOps workflow.

However, building a reliable ECS control loop currently has a state-observation problem. Today, an external controller generally has to periodically call `DescribeServices` and compare the full configuration against its desired state.

At scale, this creates several problems:
* Polling load increases with the number of services even when nothing changes.
* Frequent polling creates unnecessary API calls and potential throttling pressure.
* Infrequent polling increases drift-detection latency.
* There is no simple, cheap "has this service changed since revision X?" primitive.

**Why is this different from existing ECS EventBridge events?**

ECS EventBridge events are excellent for answering: *"What happened to the workload?"*
For example: A deployment started / A deployment completed / A task started / A task stopped.

A control loop also needs to answer a different question: *"Has the desired configuration of this service changed since I last observed it?"*

These are different requirements:
* ECS lifecycle events -> *"What happened?"*
* Desired-state revision -> *"Did the desired configuration change?"*

While we can route `UpdateService` CloudTrail events to EventBridge, it fails as a reliable control loop primitive. CloudTrail events can experience delivery latency, and more importantly, they are transient. If the controller goes offline, it has no resumable sequence to know what `UpdateService` calls it missed, forcing it to fall back to the heavy periodic full-state polling sweep we are trying to avoid.

The simplest implementation would be a monotonically increasing revision associated with the service's desired configuration (or a deterministic ETag/Hash).

Conceptually:
```json
{
"serviceArn": "arn:aws:ecs:...",
"desiredStateRevision": 1837
}
```

Contributor guide

Open the contributing guide

Research direction

Start by comparing the ECS DescribeServices and UpdateService APIs with the existing ECS EventBridge and CloudTrail events described in the issue. A complete result would provide an efficient, resumable way for external controllers to detect desired-configuration changes without repeatedly diffing full service state.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.