hashicorp / hashicorp/vault-helm

Injector pod stuck in pending state when upgraded from 0.9.0 to 0.13.0

Open
#561 2 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Shell
Stars
1.3k
Forks
898
Avg merge
3d 1h
Merged PRs (30d)
1

Description

**Describe the bug**
On 1 Node kubernetes cluster, if you upgrade the vault from 0.9.0 to 0.13.0, the agent injector pod is stuck in pending state

**To Reproduce**
Steps to reproduce the behavior:
1. Install vault 0.9.0 on 1 node Kubernetes cluster
2. Try upgrading to 0.13.0

![image](https://user-images.githubusercontent.com/36413199/124076004-4b9b0480-da63-11eb-9f03-e5cef15f9b5b.png)

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling default-scheduler 0/1 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity.
Warning FailedScheduling default-scheduler 0/1 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity.

Other useful info to include: vault pod logs, `kubectl describe statefulset vault` and `kubectl get statefulset vault -o yaml` output

**Expected behavior**
Should be able to upgrade from 0.9.0 to 0.13.0

**Additional context**
Seems that because of this check in
https://github.com/hashicorp/vault-helm/commit/53f31be2056364ae7a8c35c334f6bca0eac75de3#diff-8377b3e3740a3fcd9f682e5fb55425f2fdbece1791854b9e5013e7f1a5e60e7e

Kubernetes is looking for an node where web hook component is not present. Since this is 1 node, it's always stuck in pending state.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the affinity/anti-affinity change in commit 53f31be2056364ae7a8c35c334f6bca0eac75de3 and reproduce the 0.9.0-to-0.13.0 upgrade on a single-node Kubernetes cluster. Use the listed kubectl describe and YAML commands to confirm why the injector remains Pending; done means the upgrade succeeds and the injector pod schedules.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.