fail to scale EKS anywhere cluster for bare metal
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 328
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 9
Description
**What happened**:
I try to scale my cluster (from 1 cp node to 3 cp nodes out of 4 cp nodes, hardware.csv has 4 nodes, and I want to use 3 of them as cp nodes) with command:
```
eksctl anywhere upgrade cluster
-f cluster.yaml
--hardware-csv hardware.csv
--kubeconfig mgmt/mgmt-eks-a-cluster.kubeconfig
```
If I keep cp count 1 in the cluster.yaml, the command completes in 2 minutes, however, nothing change, even I don't see new hardware with follow command:
`kubectl get hardware -n eksa-system --show-labels `
If I change cp count to 3 in the cluster.yaml, I keep seeing this message:
```
2024-02-15T17:26:49.783Z V6 Executing command {"cmd": "/usr/bin/docker exec -i eksa_1708017136282361003 kubectl get --ignore-not-found -o json --kubeconfig mgmt/mgmt-eks-a-cluster.kubeconfig Cluster.v1alpha1.anywhere.eks.amazonaws.com --namespace default mgmt02"}
2024-02-15T17:26:49.884Z V9 Cluster generation and observedGeneration {"Generation": 2, "ObservedGeneration": 1}
2024-02-15T17:26:49.884Z V5 Error happened during retry {"error": "cluster generation (2) and observedGeneration (1) differ", "retries": 782}
2024-02-15T17:26:49.884Z V5 Sleeping before next retry {"time": "1s"}
```
**What you expected to happen**:
I expect the command to succeed with new hardware added
**How to reproduce it (as minimally and precisely as possible)**:
**Anything else we need to know?**:
**Environment**:
- EKS Anywhere Release: v0.18.2
- EKS Distro Release: 1.26
Contributor guide
Research direction
Start with the upgrade cluster command and the provided cluster.yaml and hardware.csv, then inspect the kubectl hardware output and retry logs. Determine why scaling from one to three control-plane nodes leaves hardware unchanged or keeps Cluster generation ahead of observedGeneration; done means the upgrade succeeds and new hardware is added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, go
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100