devantler-tech / devantler-tech/ksail
An out-of-band-deleted EKS cluster can no longer be cleared through the local API
- Dominant language
- Go
- Stars
- 165
- Forks
- 12
- Avg merge
- 5h 51m
- Merged PRs (30d)
- 347
Description
> 🤖 Generated by the Agentic Engineer
## Evidence
#6434 switches on the immutable-ownership guard for local API EKS mutations (part of #6203). One
consequence, surfaced while self-reviewing that PR and stated on it rather than left to be
discovered:
`runDelete` deliberately treats `clustererr.ErrClusterNotFound` as success and proceeds to clean up
local state. Its own comment records why:
> Deleting must be idempotent. ErrClusterNotFound means there is nothing to delete — a failed create
> that produced no cluster, a cluster removed out-of-band, or a race where it vanished between
> enumeration and Delete. […] otherwise the job stays pinned Failed and the UI row can never be
> dismissed (forcing an app restart or a fallback to `ksail cluster delete --name`).
The guard now runs **before** that, and verification requires `DescribeCluster` to succeed. For an
EKS cluster that has genuinely vanished from AWS — deleted in the console, or removed by another
operator — the guard fails and the delete never reaches its idempotent cleanup.
The standalone CLI path (`ensureAWSClusterManaged`) has the same property, so the two surfaces are
now consistent; this is not a regression *relative to the CLI*. It is a real reduction in what the
web UI can recover from, and it makes one existing message inaccurate: the create-refusal in
`distconfig.go` promises `ksail cluster delete --name ` "succeeds even when the cluster is already
gone", which is now weaker than its wording for EKS.
## Affected audience and impact
An operator whose EKS cluster was removed out-of-band is left with a Failed row they cannot dismiss
and local state (`spec.json`, `eks.yaml`, nodegroup snapshot) they cannot clear through either
surface. No data is at risk and nothing is silently wrong — the failure is loud — but the recovery
story is a dead end, and the message that names the way out no longer describes one.
## Expected behaviour
A genuinely-absent cluster should be distinguishable from an unverifiable one, and only the first
should permit local-state cleanup. Fail-closed must remain the default: "cannot describe the
cluster" is otherwise indistinguishable from a credential pointed at the wrong account, an expired
session, or a transient AWS error, and that ambiguity is exactly what #6203 asks the backend to
refuse on.
## Acceptance criteria
- [ ] The identity client distinguishes **cluster-does-not-exist** (AWS `ResourceNotFoundException`,
and the existing `eks.ErrClusterNotFound` empty-payload case) from every other describe failure,
as a typed result rather than string matching.
- [ ] Only that distinguished absence permits local-state cleanup; a credential, permission,
throttling, or network failure still refuses. A test covers each arm, including the
wrong-account case, so the carve-out cannot widen into "any describe error means gone".
- [ ] The absence path still refuses anything that would touch AWS — it cleans local state only.
- [ ] The `distconfig.go` create-refusal message is re-checked against whatever recovery path is
chosen, so it does not promise something that no longer works.
- [ ] Decide explicitly whether the same treatment applies to start/stop, or only to delete —
scaling a nodegroup on an absent cluster has no cleanup semantics to preserve.
Follow-up to #6434. Related: #6203 (criterion 3), #6202.
Contributor guide
Research direction
Start with runDelete, the identity client’s DescribeCluster handling, and distconfig.go, then inspect the existing EKS error and cleanup tests. Define a typed not-found result, preserve refusal for other describe failures including wrong-account access, and verify that only the absent-cluster path cleans local state without touching AWS; also resolve the start/stop scope and re-check the recovery message.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go, kubernetes
- Domain
- api, cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 56/100