aws / aws/eks-anywhere

Proper procedures for recovering EKS-A cluster from broken state.

Open
#9,968 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
2.1k
Forks
328
Avg merge
1d 4h
Merged PRs (30d)
9

Description

I have a question about recovering EKS-A provisioned clusters from a broken state.

Suppose a cluster has failed machines in both the control plane and worker nodes, and these machines are assumed to be unrecoverable (=physically broken and have to add new baremetal machines). How should this be handled when we want to execute cluster upgrade?

example@example-admin:~$ kubectl get nodes
NAME                STATUS                        ROLES           AGE    VERSION
example-cp3-26       Ready                         control-plane   191d   v1.28.15
example-cp3-27       NotReady,SchedulingDisabled   control-plane   191d   v1.29.13
example-cp5-26       Ready                         control-plane   191d   v1.28.15
example-gpu-wk3-9    NotReady,SchedulingDisabled   <none>          191d   v1.29.13
example-gpu-wk5-11   Ready                         <none>          191d   v1.28.15

Are we expected to manually add healthy control plane and worker nodes to proceed with the cluster upgrade?
Or are we expected to re-provision the cluster from scratch and execute backups?

I’m trying to understand the intended recovery path when the cluster is in an unstable state and cannot be restored using the originally provisioned machines.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are mentioned. Start by reviewing the EKS-A cluster upgrade and machine replacement procedures; done would be a maintainer-confirmed recovery path for clusters with unrecoverable control-plane and worker machines.

Written by the indexing model from the issue text.

Assessment

Domain
cloud, devops, infrastructure
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.