hashicorp / hashicorp/vault-helm
Vault using postgresql database, sometimes not un-initialises the vault by truncating vault_kv_store and vault_ha_locks
- Dominant language
- Shell
- Stars
- 1.3k
- Forks
- 898
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 1
Description
Vault using postgresql database.
Most of the time by truncating vault_kv_store and vault_ha_locks from psql db un-initialises the vault.
But randomly/sometimes it does not un-initialises the vault by truncating vault_kv_store and vault_ha_locks .
Even after truncate the vault stays in sealed state and and initialise mode. In this case we are getting stucked into issues.
**Describe the bug**
A clear and concise description of what the bug is.
- We have written logic for vault initialise/seal/unseal.
In this case once we get into similar situation we are endup with having errors and data loss.
**To Reproduce**
Steps to reproduce the behavior:
1. Install chart
2. Run vault command psql db
```
truncate vault_kv_store;
truncate vault_ha_locks;
```
Note: (sometimes only)
3. Restart the vault pod and booommm, sidecar container(our logic)api will fail
4. See error (vault logs, etc.)

**Expected behavior**
It should un-initialise the vault instead taking vault in sealed state after truncating vault_kv_store and vault_ha_locks
ted to happen.
**Environment**
* vault-helm version: 0.16.1
Chart values:
```yaml
# Paste your user-supplied values here (`helm get values `).
# Be sure to scrub any sensitive values!
## Vault
vault:
enabled: true
global:
imagePullSecrets: []
# - name: image-pull-secret
injector:
# Must be disabled
enabled: false
server:
image:
repository: "releases-docker.jfrog.io/hashicorp/vault"
tag: 1.8.6
# resources: {}
# requests:
# memory: 256Mi
# cpu: 250m
# limits:
# memory: 256Mi
# cpu: 250m
# Affinity Settings
# Commenting out or setting as empty the affinity variable, will allow
# deployment to single node services such as Minikube
affinity: |
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app.kubernetes.io/name: {{ template "vault.name" . }}
app.kubernetes.io/instance: "{{ .Release.Name }}"
component: server
topologyKey: kubernetes.io/hostname
# Toleration Settings for server pods
# This should be a multi-line string matching the Toleration array
# in a PodSpec.
tolerations: null
# Enables network policy for server pods
networkPolicy:
enabled: false
egress: []
# egress:
# - to:
# - ipBlock:
# cidr: 10.0.0.0/24
# ports:
# - protocol: TCP
# port: 443
service:
port: 8200
# Set Pipelines vault-init sidecar container
extraContainers:
- name: vault-init
image: releases-docker.jfrog.io/jfrog/pipelines-vault-init:1.20.4
imagePullPolicy: IfNotPresent
env:
- name: CHECK_INTERVAL
value: "10s"
- name: VAULT_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: VAULT_ADDRESS
value: "http://localhost:8200"
resources:
requests:
memory: 10Mi
cpu: 10m
limits:
memory: 50Mi
cpu: 50m
# Pass path of the storage config file with an external Postgres setup
extraArgs: "-config=/vault/userconfig/vault-storage-config/config.hcl"
# Secret name with the storage config file for Postgres connection
# It will be exposed to Vault in the path "/vault/userconfig/vault-storage-config/"
extraVolumes:
- type: secret
name: vault-storage-config
dataStorage:
## Must be disabled
enabled: false
# `updateStrategyType` must be set to `OnDelete` update strategy for HA setup
# https://www.vaultproject.io/docs/platform/k8s/helm/run#upgrading-vault-on-kubernetes
# It is critical to use OnDelete instead of RollingUpdate because standbys must be updated
# before the active primary. A failover to an older version of Vault must always be avoided.
updateStrategyType: "RollingUpdate"
ha:
enabled: false
replicas: 1
config: |
ui = false
listener "tcp" {
tls_disable = 1
address = "[::]:8200"
cluster_address = "[::]:8201"
}
disable_mlock = true
service_registration "kubernetes" {}
livenessProbe:
enabled: true
path: "/v1/sys/health?standbyok=true"
initialDelaySeconds: 60
serviceAccount:
create: true
name: vault
## Vault Postgres and storage config using existing secret
# It will be exposed to Vault in the path "/vault/userconfig/vault-storage-config/"
# existingSecret: vault-storage-config
## Role Based Access for vault-init container
## Ref: https://kubernetes.io/docs/admin/authorization/rbac/
rbac:
role:
## Rules to create. It follows the role specification
rules:
- apiGroups:
- ''
resources:
- secrets
verbs:
- "*"
```
```
Secret: vault-storage-config
config.hcl: | │
│ storage "postgresql" { │
│ connection_url = "postgres://apiuser:pass@pipelines-postgresql:5432/pipelinesdb?sslmode=disable" │
│ ha_enabled = "false" │
│ } │
│ postgresql-url: postgres://apiuser:pass@pipelines-postgresql:5432/pipelinesdb?sslmode=disable
```
**Additional context**
We have been facing many issues, please fix this asap
Contributor guide
Research direction
No repository file or test is identified. Start by reproducing the chart's PostgreSQL setup with the supplied config.hcl and truncate/restart steps, then inspect the Vault logs; done means the sequence reliably un-initialises Vault instead of leaving it sealed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes, postgresql
- Domain
- databases, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100