ClickHouse / ClickHouse/ClickStack-helm-charts
Secure ClickHouse Password Management Using Environment Variables and External Secrets
- Dominant language
- Shell
- Stars
- 75
- Forks
- 54
- Avg merge
- 3d 19h
- Merged PRs (30d)
- 7
Description
### Issue
Currently the `clickhouse.config.users.appUserPassword` and `otelUserPassword` are configured in plaintext in values.yaml (likely stored in github for config-as-code). They are then templated into [users.xml](https://github.com/hyperdxio/helm-charts/blob/38e5d05ed07145f3ed2bb807915962c593d3e9a5/charts/hdx-oss-v2/data/users.xml#L26) and stored in a [ConfigMap](https://github.com/hyperdxio/helm-charts/blob/38e5d05ed07145f3ed2bb807915962c593d3e9a5/charts/hdx-oss-v2/templates/clickhouse-deployment.yaml#L77-L79) on the cluster.
The above approach is secure if Clickhouse remains a ClusterIP, however, if the service type for Clickhouse is changed to NodePort or LoadBalancer, then it will be exposed outside of the K8s cluster. Doing so causes Clickhouse be insecure because the passwords are stored in plaintext, both in values.yaml (likely in github) and in the users.xml ConfigMap (not a secure K8s Secret).
### Proposed Solution
Add support for external secrets while maintaining backward compatibility with the current inline password approach.
### Recommended Changes
1. Update `values.yaml`
Add external secret configuration options:
```yaml
clickhouse:
config:
users:
# Existing inline password support (for backward compatibility)
appUserPassword: "hyperdx"
otelUserPassword: "otelcollectorpass"
otelUserName: "otelcollector"
# NEW: External secret support
useExternalSecret: false
externalSecret:
name: ""
appUserPasswordKey: "app-user-password"
otelUserPasswordKey: "otel-user-password"
```
2. Modify ClickHouse Deployment `clickhouse-deployment.yaml`
Add environment variables that read from secrets when external secrets are enabled:
```yaml
# Add to the ClickHouse container env section:
env:
- name: CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT
value: "1"
{{- if .Values.clickhouse.config.users.useExternalSecret }}
- name: CLICKHOUSE_APP_USER_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.appUserPasswordKey }}
- name: CLICKHOUSE_OTEL_USER_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.otelUserPasswordKey }}
{{- end }}
```
3. Update `users.xml`
Modify to support both password methods:
```xml
{{- if .Values.clickhouse.config.users.useExternalSecret }}
{{- else }}
{{ .Values.clickhouse.config.users.appUserPassword }}
{{- end }}
<{{ .Values.otel.clickhouseUser | default .Values.clickhouse.config.users.otelUserName }}>
{{- if .Values.clickhouse.config.users.useExternalSecret }}
{{- else }}
{{ .Values.otel.clickhousePassword | default .Values.clickhouse.config.users.otelUserPassword }}
{{- end }}
```
4. Update OTEL collector Deployment `otel-collector-deployment.yaml`
Replace the plain text password with secret reference:
```yaml
# Replace the existing CLICKHOUSE_PASSWORD env var:
- name: CLICKHOUSE_PASSWORD
{{- if .Values.clickhouse.config.users.useExternalSecret }}
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.otelUserPasswordKey }}
{{- else }}
value: {{ .Values.otel.clickhousePassword | default .Values.clickhouse.config.users.otelUserPassword }}
{{- end }}
```
5. Remove the now unused secrets.yaml
```yaml
# DELETE this entire secret block:
---
apiVersion: v1
kind: Secret
metadata:
name: {{ include "hdx-oss.fullname" . }}-clickhouse-secrets
labels:
{{- include "hdx-oss.labels" . | nindent 4 }}
data:
appUserPassword: {{ .Values.clickhouse.config.users.appUserPassword | toString | b64enc }}
otelUserPassword: {{ .Values.clickhouse.config.users.otelUserPassword | toString | b64enc }}
```
6. Update Default Connections `values.yaml`
Modify the default connections to support external secrets:
```yaml
defaultConnections: |
[
{
"name": "Local ClickHouse",
"host": "http://{{ include "hdx-oss.fullname" . }}-clickhouse:8123",
"port": 8123,
"username": "app",
{{- if .Values.clickhouse.config.users.useExternalSecret }}
"passwordFromSecret": {
"name": "{{ .Values.clickhouse.config.users.externalSecret.name }}",
"key": "{{ .Values.clickhouse.config.users.externalSecret.appUserPasswordKey }}"
}
{{- else }}
"password": "{{ .Values.clickhouse.config.users.appUserPassword }}"
{{- end }}
}
]
```
7. Update documentation
Add examples in the chart's README or values.yaml comments showing:
```yaml
# Example usage with external secrets:
clickhouse:
config:
users:
useExternalSecret: true
externalSecret:
name: "my-clickhouse-passwords"
appUserPasswordKey: "app-password"
otelUserPasswordKey: "otel-password"
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.