aws-sigv4-proxy-admission-controller's webhook can result in entire cluster being inoperable
- Dominant language
- Mustache
- Stars
- 1.3k
- Forks
- 1.1k
- Avg merge
- 22m
- Merged PRs (30d)
- 5
Description
**Describe the bug**
aws-sigv4-proxy-admission-controller's webhook doesn't handle outages well.
1. When webhook fails, it fails request (`failurePolicy` is not `Ignore`).
2. Webhook is triggered on controller itself and every other pod in cluster.
Both of these issues combined result in failure to start controller due to its own hook failing because controller is not ready. Circular dependency.
More than that, webhook is triggered when you add new nodes to the cluster. If controller is inoperable, this results in failed nodes (they are indefinitely 'not ready' and cannot join cluster). This in certain circumstances can result in entire cluster being inoperable.
**Steps to reproduce**
Install helm chart and depending on the order of manifests it will trigger.
It will also trigger if you delete all aws-sigv4-proxy-admission-controller instances from cluster or delete all cluster nodes and try creating them again.
**Expected outcome**
1. Controller deployment does not depend on its own hook.
2. Resources without sigv4 sidecar annotations do not require working aws-sigv4-proxy-admission-controller to deploy.
**Environment**
* Chart name: aws-sigv4-proxy-admission-controller
* Chart version: 0.1.2
* Kubernetes version: 1.20 and 1.21
* Using EKS (yes/no), if so version? 1.20 and 1.21
**Additional Context**:
Contributor guide
Research direction
Start by inspecting the aws-sigv4-proxy-admission-controller Helm chart, its webhook configuration, and the controller deployment. Reproduce installation-order and controller/node deletion scenarios on Kubernetes 1.20 or 1.21, then trace which resources match the webhook. Done means the controller does not trigger its own hook and unannotated resources can deploy when the controller is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100