aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(cdk): tag-safe strategy — exclude replacement-sensitive resources from volatile github:* tags

Open
#229 0 comments 0 reactions 0 assignees View on GitHub
enhancement infra-cdk
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

## Problem

All `github:*` tags change on every deployment (`sha`, `run-id`, `actor`, etc.). Some CloudFormation resource types treat **any tag change** as requiring replacement (new physical resource ID). When a replaced resource has dependents with uniqueness constraints (e.g., Route53 Resolver VPC association), the cascade fails and corrupts the CloudFormation state — requiring manual intervention to recover.

**Incident history:**
- PR #211 / #221: `CfnResolverQueryLoggingConfig` tag update → replacement → Association fails on VPC uniqueness → stack stuck in `UPDATE_ROLLBACK_COMPLETE`
- Current deploy: Association still references stale Config ID from prior failed replacement

## Desired outcome ("no touch needed")

Deploys should never fail due to tag propagation triggering resource replacement. The solution should be self-maintaining — new resources added to the stack should be automatically protected without manual discovery.

## Options

### Option A: CDK Aspect that auto-excludes replacement-sensitive resources (Recommended)

Create a custom Aspect that runs at synth time and inspects each `CfnResource`. If the resource's CloudFormation spec indicates that tags require replacement (`Update requires: Replacement` on the `Tags` property), auto-exclude it from tagging.

**Pros:** Self-maintaining, no manual denylist, protects future resources automatically.
**Cons:** Depends on CloudFormation Resource Spec accuracy (some resources don't report correctly).

### Option B: Reactive denylist with synth-time validation (Fallback)

Maintain `excludeResourceTypes` array (current approach), but add a synth-time check: a CDK Aspect that warns or fails if a tagged resource is known to treat tags as replacement. Build the denylist from:
1. Known problematic types (Route53Resolver, etc.)
2. CloudFormation Resource Spec where available
3. Discovered failures added incrementally

**Pros:** Simple, explicit, doesn't rely on spec accuracy.
**Cons:** Reactive — new problematic types discovered via deploy failures.

### Option C: Hybrid (A + B)

Use the Aspect from Option A as the primary gate, with an explicit denylist (Option B) as override for resources where the spec is inaccurate or missing. Log warnings at synth time when a resource is excluded by either mechanism.

## Immediate fix needed

Add `AWS::Route53Resolver::ResolverQueryLoggingConfigAssociation` to the existing `excludeResourceTypes` array to unblock deploys. Then comment out the Association, deploy (clears CF state), uncomment, redeploy (fresh creation).

## Acceptance criteria

- [ ] Deploys with changed `github:*` tags never trigger resource replacement
- [ ] New replacement-sensitive resources are automatically detected (Aspect or CI check)
- [ ] Existing broken CF state is recovered (one-time manual fix documented)
- [ ] Approach handles resources where CloudFormation spec doesn't accurately report replacement behavior

## References

- PR #211: Per-session IAM scoping (introduced tags on all resources)
- PR #221: Excluded `CfnResolverQueryLoggingConfig` from tags
- [CloudFormation Resource Spec](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html)
- `cdk/src/main.ts` lines 45-67: tag application logic

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.