aws / aws/eks-anywhere

gitOps upgrade failed due to diskGiB field is immutable

Open
#4,339 5 comments 0 reactions 0 assignees View on GitHub
area/docs external
Dominant language
Go
Stars
2.1k
Forks
328
Avg merge
1d 4h
Merged PRs (30d)
9

Description

**What happened**:

Cluster yaml file in git repo was edited to add more disk space

From
```
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: VSphereMachineConfig
metadata:
annotations:
anywhere.eks.amazonaws.com/etcd: "true"
name: k8s-lower-wlr-machines
namespace: default
spec:
diskGiB: 20
.....

```
Change to this

```
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: VSphereMachineConfig
metadata:
annotations:
anywhere.eks.amazonaws.com/etcd: "true"
name: k8s-lower-wlr-machines
namespace: default
spec:
diskGiB: 50
.....

```

Nothing happened.

flux-system kustomize-controller pod log shows this

`{"level":"error","ts":"2022-12-08T19:36:50.911Z","logger":"controller.kustomization","msg":"Reconciliation failed after 858.979712ms, next try in 10m0s","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"flux-system","namespace":"flux-system","revision":"main/f8993e13defd7211d3dab445bb51d2441a5169fd","error":"VSphereMachineConfig/default/k8s-lower-wlr-machines dry-run failed, reason: Invalid, error: admission webhook \"validation.vspheremachineconfig.anywhere.amazonaws.com\" denied the request: VSphereMachineConfig.anywhere.eks.amazonaws.com \"k8s-lower-wlr-machines\" is invalid: spec.diskGiB: Forbidden: field is immutable\n"}
`

Manually upgrade with eksctl anywhere command completed successfully without any issue.

**What you expected to happen**:

gitOps should be able to upgrade the cluster on this field.

**Environment**:
- EKS Anywhere Release: v0.12.0
- EKS Distro Release: 1.22-eks-11

Contributor guide

Open the contributing guide

Research direction

Start with the VSphereMachineConfig validation webhook and the flux-system kustomize-controller error shown in the report, then compare the GitOps upgrade path with the successful eksctl anywhere command. Reproduce the diskGiB change using the provided cluster YAML and release versions. Done means GitOps can apply this diskGiB update without the immutable-field failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.