bitnami / bitnami/sealed-secrets

Controller restart loop on first install of the Helm chart

Open
#697 7 comments 1 reaction 0 assignees View on GitHub
backlog enhancement
Dominant language
Go
Stars
9.3k
Forks
776
Avg merge
1d 13h
Merged PRs (30d)
30

Description

**Which component**:
Controller / Helm Chart Deployment.yaml

**Is your feature request related to a problem? Please describe.**
On our latest deployments on several private AKS 1.21 clusters, all initial deployments fail. This is caused by the way the liveness probe is setup.
The liveness probe checks the /healthz endpoint, but this is only available when the HTTP server has started.
Before HTTP server startup, the controller is checking if a private key secret already exists. On first install this is never the case and it will generate a private key.

For some reason this process of checking the secret & generating a private key takes more time than what the default livenessprobe configuration allows. As the livenessprobe will trigger restarts before the key is created, the controller is never able to start and stay alive.

**Describe the solution you'd like**
A startup probe can be used for mitigating this issue. Or some more control over the configuration of the liveness probe itself, it uses kubernetes defaults with no option to change the values.

**Describe alternatives you've considered**
-

**Additional context**
-

Contributor guide

Open the contributing guide

Research direction

Inspect the Helm chart Deployment.yaml and the controller's /healthz startup path, focusing on the timing of private-key generation during a first install. Reproduce an initial deployment with the default probe settings and verify that the controller remains running through key creation and becomes healthy.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, helm, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.