ClickHouse / ClickHouse/ClickStack-helm-charts

Secure ClickHouse Password Management Using Environment Variables and External Secrets

Open
#125 3 comments 7 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
75
Forks
54
Avg merge
3d 19h
Merged PRs (30d)
7

Description

### Issue

Currently the `clickhouse.config.users.appUserPassword` and `otelUserPassword` are configured in plaintext in values.yaml (likely stored in github for config-as-code). They are then templated into [users.xml](https://github.com/hyperdxio/helm-charts/blob/38e5d05ed07145f3ed2bb807915962c593d3e9a5/charts/hdx-oss-v2/data/users.xml#L26) and stored in a [ConfigMap](https://github.com/hyperdxio/helm-charts/blob/38e5d05ed07145f3ed2bb807915962c593d3e9a5/charts/hdx-oss-v2/templates/clickhouse-deployment.yaml#L77-L79) on the cluster.

The above approach is secure if Clickhouse remains a ClusterIP, however, if the service type for Clickhouse is changed to NodePort or LoadBalancer, then it will be exposed outside of the K8s cluster. Doing so causes Clickhouse be insecure because the passwords are stored in plaintext, both in values.yaml (likely in github) and in the users.xml ConfigMap (not a secure K8s Secret).

### Proposed Solution
Add support for external secrets while maintaining backward compatibility with the current inline password approach.

### Recommended Changes

1. Update `values.yaml`
Add external secret configuration options:
```yaml
clickhouse:
config:
users:
# Existing inline password support (for backward compatibility)
appUserPassword: "hyperdx"
otelUserPassword: "otelcollectorpass"
otelUserName: "otelcollector"

# NEW: External secret support
useExternalSecret: false
externalSecret:
name: ""
appUserPasswordKey: "app-user-password"
otelUserPasswordKey: "otel-user-password"
```

2. Modify ClickHouse Deployment `clickhouse-deployment.yaml`
Add environment variables that read from secrets when external secrets are enabled:
```yaml
# Add to the ClickHouse container env section:
env:
- name: CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT
value: "1"
{{- if .Values.clickhouse.config.users.useExternalSecret }}
- name: CLICKHOUSE_APP_USER_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.appUserPasswordKey }}
- name: CLICKHOUSE_OTEL_USER_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.otelUserPasswordKey }}
{{- end }}
```

3. Update `users.xml`
Modify to support both password methods:
```xml

{{- if .Values.clickhouse.config.users.useExternalSecret }}

{{- else }}
{{ .Values.clickhouse.config.users.appUserPassword }}
{{- end }}

<{{ .Values.otel.clickhouseUser | default .Values.clickhouse.config.users.otelUserName }}>
{{- if .Values.clickhouse.config.users.useExternalSecret }}

{{- else }}
{{ .Values.otel.clickhousePassword | default .Values.clickhouse.config.users.otelUserPassword }}
{{- end }}

```

4. Update OTEL collector Deployment `otel-collector-deployment.yaml`
Replace the plain text password with secret reference:
```yaml
# Replace the existing CLICKHOUSE_PASSWORD env var:
- name: CLICKHOUSE_PASSWORD
{{- if .Values.clickhouse.config.users.useExternalSecret }}
valueFrom:
secretKeyRef:
name: {{ .Values.clickhouse.config.users.externalSecret.name }}
key: {{ .Values.clickhouse.config.users.externalSecret.otelUserPasswordKey }}
{{- else }}
value: {{ .Values.otel.clickhousePassword | default .Values.clickhouse.config.users.otelUserPassword }}
{{- end }}
```

5. Remove the now unused secrets.yaml
```yaml
# DELETE this entire secret block:
---
apiVersion: v1
kind: Secret
metadata:
name: {{ include "hdx-oss.fullname" . }}-clickhouse-secrets
labels:
{{- include "hdx-oss.labels" . | nindent 4 }}
data:
appUserPassword: {{ .Values.clickhouse.config.users.appUserPassword | toString | b64enc }}
otelUserPassword: {{ .Values.clickhouse.config.users.otelUserPassword | toString | b64enc }}
```

6. Update Default Connections `values.yaml`
Modify the default connections to support external secrets:
```yaml
defaultConnections: |
[
{
"name": "Local ClickHouse",
"host": "http://{{ include "hdx-oss.fullname" . }}-clickhouse:8123",
"port": 8123,
"username": "app",
{{- if .Values.clickhouse.config.users.useExternalSecret }}
"passwordFromSecret": {
"name": "{{ .Values.clickhouse.config.users.externalSecret.name }}",
"key": "{{ .Values.clickhouse.config.users.externalSecret.appUserPasswordKey }}"
}
{{- else }}
"password": "{{ .Values.clickhouse.config.users.appUserPassword }}"
{{- end }}
}
]
```

7. Update documentation
Add examples in the chart's README or values.yaml comments showing:
```yaml
# Example usage with external secrets:
clickhouse:
config:
users:
useExternalSecret: true
externalSecret:
name: "my-clickhouse-passwords"
appUserPasswordKey: "app-password"
otelUserPasswordKey: "otel-password"
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.