Starting EKS Anywhere on VMware, etcd fails to start
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 328
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 9
Description
**What happened**:
tmcooper@ubuntu-server-2404:~$ eksctl anywhere create cluster -f eksa-w01-cluster.yaml
Warning: VSphereDatacenterConfig configured in insecure mode
Performing setup and validations
Warning: VSphereDatacenterConfig configured in insecure mode
✅ Connected to server
✅ Authenticated to vSphere
✅ Datacenter validated
✅ Network validated
✅ Datastore validated
✅ Folder validated
✅ Resource pool validated
✅ Datastore validated
✅ Folder validated
✅ Resource pool validated
✅ Datastore validated
✅ Folder validated
✅ Resource pool validated
✅ Machine config tags validated
✅ Control plane and Workload templates validated
✅ administrator@vsphere.local user vSphere privileges validated
✅ Vsphere Provider setup is valid
✅ Validate OS is compatible with registry mirror configuration
✅ Validate certificate for registry mirror
✅ Validate authentication for git provider
✅ Validate cluster's eksaVersion matches EKS-A version
✅ Validate cluster's kubelet configuration for Bottlerocket OS
✅ Validate cluster's worker node kubelet configuration for Bottlerocket OS
Creating new bootstrap cluster
Provider specific pre-capi-install-setup on bootstrap cluster
Installing cluster-api providers on bootstrap cluster
Provider specific post-setup
Installing EKS-A custom components on bootstrap cluster
Installing EKS-D components
Installing EKS-A custom components (CRD and controller)
Creating new management cluster
(gets hung)
From the log where it is repeating the problem:
{"T":1733494047021445202,"M":"Sleeping before next retry","time":"1s"}
{"T":1733494048021896267,"M":"Executing command","cmd":"/usr/bin/docker exec -i eksa_1733493551451751668 kubectl get --ignore-not-found -o json --kubeconfig kn01/generated/kn01.kind.kubeconfig Cluster.v1alpha1.anywhere.eks.amazonaws.com --namespace default kn01"}
{"T":1733494048518935993,"M":"Cluster generation and observedGeneration","Generation":1,"ObservedGeneration":1}
{"T":1733494048519028611,"M":"Error happened during retry","error":"cluster condition ControlPlaneReady is False: Etcd is not ready","retries":59}
{"T":1733494048519061258,"M":"Sleeping before next retry","time":"1s"}
_Why is etcd not becoming ready and does it have some log?_
The file with the YAML configuration is attached.
**What you expected to happen**:
My EKS Kubernetes to be set up
**How to reproduce it (as minimally and precisely as possible)**:
Set up eks anywhere, docker,
set environment passwords for VMWare
export EKSA_VSPHERE_USERNAME=administrator@vsphere.local
export EKSA_VSPHERE_PASSWORD=
run
eksctl anywhere create cluster -f eksa-w01-cluster.yaml
**Anything else we need to know?**:
**Environment**:
- Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-49-generic x86_64)
- eksctl 0.197.0
- VMWare:
Client version: 2.14.0
Client build number: 21993070
ESXi version: 8.0.2
ESXi build number: 22380479
- Vcenter 8
- EKS Anywhere Release: 0.197.0
- EKS Distro Release:
- kubectl version
Client Version: v1.31.3
Kustomize Version: v5.4.2
Contributor guide
Research direction
Start by reproducing the failure with eksctl anywhere create cluster -f eksa-w01-cluster.yaml and inspect the retry output for the ControlPlaneReady and Etcd conditions. Find the management-cluster and etcd logs associated with the repeated kubectl get command; done means the management cluster completes creation with etcd becoming ready.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go, kubernetes
- Domain
- distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100