ptd ensure --destroy fails when deprovisioning a cluster and requires manual intervention
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 7
- Forks
- 2
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 3
Description
Trying to delete eval01-production cluster because it was long-overdue to be deleted and was costing money and causing alert noise.
We have the start of a document here: https://positpbc.atlassian.net/wiki/spaces/PTD/pages/1510015005/Decommissioning+Internal+PTD+Resources
but it is incomplete.
Running ptd ensure eval01-production --destroy started deleting from the persistent_reprise step - it seems like this might not be the correct order of operations that things need to be deleted in.
Re-running the commands several times, I also encountered errors like:
Diagnostics:
aws:iam:OpenIdConnectProvider (eval01-production-20250422):
error: resource "urn:pulumi:eval01-production::ptd-aws-workload-eks::ptd:AWSWorkloadEKS$ptd:AWSEKSCluster$aws:eks/cluster:Cluster$aws:iam/openIdConnectProvider:OpenIdConnectProvider::eval01-production-20250422" cannot be deleted
because it is protected. To unprotect the resource, either remove the `protect` flag from the resource in your Pulumi program and run `pulumi up`, or use the command:
`pulumi state unprotect 'urn:pulumi:eval01-production::ptd-aws-workload-eks::ptd:AWSWorkloadEKS$ptd:AWSEKSCluster$aws:eks/cluster:Cluster$aws:iam/openIdConnectProvider:OpenIdConnectProvider::eval01-production-20250422'`
We need a reliable playbook describing the steps to delete a cluster, including RDS snapshots, FSx snapshots, S3 buckets with chronicle data, and so on.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ptd ensure eval01-production --destroy workflow and the linked Decommissioning Internal PTD Resources document. Review the persistent_reprise step and the Pulumi protected-resource diagnostic, then map the required handling for RDS snapshots, FSx snapshots, and S3 buckets containing chronicle data. Done means a reliable, ordered cluster-deletion playbook covers these resources and manual-intervention cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- cli, cloud, devops, documentation, infrastructure
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100