aws-samples / aws-samples/sample-collaborative-ai-dlc
Digest-aware ECR cleanup for core, managed-environment, and managed-tool images
- Dominant language
- JavaScript
- Stars
- 75
- Forks
- 23
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 24
Description
## Context
PR #397 removed `aws_ecr_lifecycle_policy.agentcore` (previously "keep only the last 3 images") because a blind count-based expiry is no longer safe once managed build environments exist. Raised in review by @jeromevdl on `terraform/modules/compute/agentcore/main.tf`.
Three repositories now hold digest-pinned images:
| Repository | Tag mutability | Tag scheme | Pinned by |
|---|---|---|---|
| `-agentcore-` | MUTABLE | `substr(sha256(agentcore files), 0, 16)` | published environment revisions and active intent snapshots, by `sha256:` digest |
| `-managed-environments-` | IMMUTABLE | `revisionId` | environment revisions; child revisions pin their base by digest |
| `-managed-tools-` | IMMUTABLE | tool image tag | catalog tool versions; environment recipes snapshot `imageUri@imageDigest` |
A count- or age-based expiry over any of these can delete a digest that a published revision or a running intent still references by `sha256:`, which breaks the immutability invariant the whole managed-environment design rests on: a revision that built and verified once must remain byte-reproducible for its whole lifetime.
## What is shipped as an interim safeguard
PR #397 adds an UNTAGGED-only expiry to the two IMMUTABLE repositories (`managed-environments`, `managed-tools`). Both are safe because every push in those repos is tagged (`image_ref=$IMAGE_REPOSITORY_URI:$IMAGE_TAG`, `tool_ref="${TOOL_REPOSITORY_URI}:${TOOL_IMAGE_TAG}"`) and IMMUTABLE tags mean a tag can never be moved off a digest — so an untagged image there is only garbage from an aborted or partial push, never something a revision pins.
The `agentcore` repository deliberately gets **no** policy, not even UNTAGGED-only: its tags are MUTABLE and derived from a content hash, so a non-reproducible rebuild under the same tag orphans the previous digest while published revisions and active intents still pin it. An UNTAGGED rule would delete exactly that digest.
## What is still needed
A registry-aware cleanup that computes the live reference set before deleting anything:
1. Enumerate referenced digests:
- `base.imageDigest` and every `tools[].imageDigest` on all environment revisions that are `PUBLISHED` (and any non-terminal build state)
- the core image digest recorded on every active intent snapshot
- the digest currently backing the deployed AgentCore runtime
2. `DescribeImages` on each repository and delete only digests absent from that set, with a grace period (e.g. minimum age of 30 days) so an in-flight build cannot lose its base mid-run.
3. Run it on a schedule (EventBridge → Lambda), dry-run by default, and emit a metric for reclaimed bytes and for any skipped-because-referenced digest.
4. Consider recording a reverse index (digest → referencing revisions/intents) at publish time so step 1 does not need a full table scan.
## Acceptance criteria
- No digest referenced by a published revision, a non-terminal revision build, or an active intent is ever deleted.
- Deletion is opt-in per environment and dry-run by default.
- A test proves that a digest pinned only by an active intent (not by any tag) survives a cleanup pass.
- The interim UNTAGGED-only policies can then be superseded or kept as a cheap first pass.
Contributor guide
Research direction
Start by reviewing PR #397 and terraform/modules/compute/agentcore/main.tf to understand the interim ECR policies and the three image repositories. Trace the published revision, non-terminal build, active intent, and deployed runtime references described in the issue, then define the cleanup flow and its safeguards. Done includes dry-run-by-default, opt-in deletion, grace-period handling, metrics, and a test showing an intent-only digest survives.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, terraform
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100