hashicorp / hashicorp/vault-secrets-operator
VaultDynamicSecret fails to be updated when Vault rotates the credential
- Dominant language
- Go
- Stars
- 600
- Forks
- 146
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 6
Description
**Describe the bug**
While using a `VaultDynamicSecret` for a Vault `database` secret engine static role with a 90 day rotation period, VSO is unable to successfully rotate the k8s secret when Vault rotates the credentials.
In this instance we're rotating the database credentials for Grafana
**To Reproduce**
Steps to reproduce the behavior:
1. Deploy Vault static role for database connection
```
vault read database/static-roles/grafana ─╯
Key Value
--- -----
credential_type password
db_name grafana-rds
last_vault_rotation 2025-01-11T19:14:13.546015704Z
rotation_period 2160h
rotation_statements []
username grafana
```
2. Deploy Grafana with `envFrom` for the k8s secret to be created by VSO as required
```yaml
envFrom:
- secretRef:
name: grafana-db-credentials
optional: false
```
3. Deploy VSO VaultDynamicSecret
```yaml
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
annotations:
database-host: :5432
name: grafana-db-credentials
namespace: monitoring
spec:
allowStaticCreds: true
destination:
create: true
name: grafana-db-credentials
overwrite: true
transformation:
excludes:
- password|username
templates:
GF_DATABASE_HOST:
text: |
{{- $grafana := get .Annotations "database-host" -}}
{{ $grafana | trim }}
GF_DATABASE_PASSWORD:
text: '{{ .Secrets.password | trim }}'
GF_DATABASE_USER:
text: '{{ .Secrets.username | trim }}'
mount: database
path: static-creds/grafana
rolloutRestartTargets:
- kind: Deployment
name: grafana
vaultAuthRef: grafana
```
The credentials are able to be retrieved manually and rotated successfully through Vault and confirmed to be operational. In fact when VSO does retrieve them and create the k8s secret it works fine. The issue is only when the credentials are rotated.
**Expected behavior**
Expect that when the credentials are rotated that VSO will update the credentials which should trigger the rolling restart and the application service should be able to connect to database and operate as normal.
**Environment**
* Kubernetes version:
* Distribution or cloud vendor (OpenShift, EKS, GKE, AKS, etc.): EKS
* Other configuration options or runtime services (istio, etc.): Vault 1.18.1
* vault-secrets-operator version: 0.9.1
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with the VaultDynamicSecret configuration shown, using a Vault database static role and the stated 90-day rotation period on VSO 0.9.1. Compare the initial credential retrieval with the post-rotation behavior, checking whether the Kubernetes Secret changes and whether the Grafana rollout restart is triggered. Done means rotated credentials reach the Secret and the application can reconnect.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100