aws / aws/containers-roadmap

[ECS] [request]: Allow images with prefer-cached to get cleaned up after a certain time

Open
#2,670 0 comments 3 reactions 0 assignees View on GitHub
ECS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

moving from https://github.com/aws/amazon-ecs-agent/issues/4678

### Community Note

- Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment

### Tell us about your request
Add a configuration option that allows time-based cleanup of cached images when `ECS_IMAGE_PULL_BEHAVIOR=prefer-cached` is enabled. Specifically, introduce an agent setting to delete cached images older than a specified age so disk usage does not grow indefinitely.

### Which service(s) is this request for?
ECS (ECS Agent on EC2/Anywhere)

### Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We deploy frequently (20–30 times/day) and want the performance benefits of `prefer-cached`. However, the current implementation disables image cleanup entirely when `prefer-cached` is set, causing old images to accumulate and eventually fill disk space. Reference: [docker_image_manager.go#L351](https://github.com/aws/amazon-ecs-agent/blob/90742cd68338075989cd24e93ff8e5b9216d788c/agent/engine/docker_image_manager.go#L351).

We’d like to keep recent images cached for fast deploys, while automatically removing images older than a configurable age (e.g., multi-day-old images) that newer deploys won’t use again.

### Are you currently working around this issue?
Partial workarounds include manual/automated `docker image prune` or instance rotation, which are disruptive and error-prone. There’s no native, policy-based cleanup when `prefer-cached` is enabled.

### Additional context
- Proposed new configuration: `ECS_PREFER_CACHED_CLEANUP_AGE` (duration)
- Behavior: When `prefer-cached` is used, allow cleanup of cached images older than the specified duration
- Example:
```bash
ECS_IMAGE_PULL_BEHAVIOR=prefer-cached
ECS_PREFER_CACHED_CLEANUP_AGE=72h # Clean up images older than 3 days
```
- This would retain the performance benefits of caching recent images while preventing disk usage from growing indefinitely.
- Happy to open a PR if this direction sounds good; open to alternative approaches as well.
- Moving from: `https://github.com/aws/amazon-ecs-agent/issues/4678`

### Attachments
N/A

Contributor guide

Open the contributing guide

Research direction

Start with agent/engine/docker_image_manager.go at the referenced cleanup logic and trace how ECS_IMAGE_PULL_BEHAVIOR=prefer-cached currently affects image cleanup. Define how ECS_PREFER_CACHED_CLEANUP_AGE should be parsed and applied, including the 72h example, then verify that recent cached images remain while older images can be removed without disrupting deployments.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.