aws / aws/eks-anywhere

Kubernetes API server does not start if DHCP assigns new IP addresses diff from originals

Open
#2,890 4 comments 0 reactions 0 assignees View on GitHub
external kind/bug priority/p1 question triage/accepted
Dominant language
Go
Stars
2.1k
Forks
328
Avg merge
1d 4h
Merged PRs (30d)
9

Description

**What happened**:
DHCP leases expired for cluster and assigned new IP addresses, Kubernetes API server would not start after, error found in journal was `kubelet.go:2451] "Error getting node" err="node \"192.168.3.192\" not found"`

**What you expected to happen**:
Cluster to detect new IP addresses assigned by DHCP and adjust accordingly so API server and other services can come up

**How to reproduce it (as minimally and precisely as possible)**:
Create a new EKSA cluster, shut it down, delete DHCP leases so NEW IP addresses get assigned, restart cluster

**Anything else we need to know?**:
NODE_IP is hard-coded with original DHCP IP address in the `env` file located under `/etc/kubernetes/kubelet`, by statically assigning the original address via DHCP, I was able to get the cluster working again.

**Environment**:
- EKS Anywhere Release: v0.10.1
- EKS Distro Release: 1.22

Contributor guide

Open the contributing guide

Research direction

Start by examining how the `/etc/kubernetes/kubelet` env file sets `NODE_IP` and compare it with the kubelet journal error showing the old address. Reproduce the issue by deleting DHCP leases and restarting the cluster; done means the API server and other services start with newly assigned addresses.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.