airbytehq / airbytehq/airbyte

Duplicate ConfigMap Keys in Airbyte Helm Chart v2.0.12

Ouverte
#66,546 9 commentaires 21 réactions 0 personnes assignées Voir sur GitHub
area/platform autoteam community needs-triage team/compose team/platform-move type/bug
Langage dominant
Python
Étoiles
22.1k
Forks
5.3k
Métriques de merge des PR
Métriques de PR en attente

Description

### Helm Chart Version

2.0.12 (Helm v2)

### What step the error happened?

Upgrading the Platform or Helm Chart

### Relevant information

## Summary

Airbyte Helm Chart v2.0.12 generates invalid Kubernetes manifests due to duplicate ConfigMap keys, causing validation failures with tools like `kubeconform`.

## Error Details

**Validation Error:**

```bash
airbyte.yaml - ConfigMap airbyte-airbyte-env failed validation: error unmarshalling resource: error converting YAML to JSON: yaml: unmarshal errors:
line 264: key "DATAPLANE_CLIENT_ID_SECRET_KEY" already set in map
line 266: key "DATAPLANE_CLIENT_SECRET_SECRET_KEY" already set in map
```

## Root Cause

Two template functions unconditionally generate identical ConfigMap keys:

### 1. Auth Bootstrap Template

**File:** [charts/v2/airbyte/templates/config/_auth.tpl](https://github.com/airbytehq/airbyte-platform/blob/main/charts/v2/airbyte/templates/config/_auth.tpl#L48-L57)

The `airbyte.auth.bootstrap.configVars` function generates:

```yaml
DATAPLANE_CLIENT_ID_SECRET_KEY: {{ include "airbyte.auth.bootstrap.dataPlane.clientIdSecretKey" . | quote }}
DATAPLANE_CLIENT_SECRET_SECRET_KEY: {{ include "airbyte.auth.bootstrap.dataPlane.clientSecretSecretKey" . | quote }}
```

### 2. WorkloadLauncher Template

File: https://github.com/airbytehq/airbyte-platform/blob/main/charts/v2/airbyte/templates/config/_workloadLauncher.tpl#L232-L237

The airbyte.workloadLauncher.dataPlane.configVars function generates:

```yaml
DATAPLANE_CLIENT_ID_SECRET_KEY: {{ include "airbyte.workloadLauncher.dataPlane.clientIdSecretKey" . | quote }}
DATAPLANE_CLIENT_SECRET_SECRET_KEY: {{ include "airbyte.workloadLauncher.dataPlane.clientSecretSecretKey" . | quote }}
```

### 3. Both Included Unconditionally

File: https://github.com/airbytehq/airbyte-platform/blob/main/charts/v2/airbyte/templates/env-configmap.yaml#L12-L54

```yaml
data:
{{- include "airbyte.auth.bootstrap.configVars" . | nindent 2 }} # Line 12 - generates DATAPLANE keys
# ... other includes ...
{{- include "airbyte.workloadLauncher.dataPlane.configVars" . | nindent 2 }} # Line 54 - duplicates same keys
```

### Impact

- Kubernetes validation fails with strict YAML parsers (kubeconform, etc.)
- CI/CD pipelines break when using proper validation tools
- Chart deployment may fail in environments with strict validation

### Reproduction

1. Use Airbyte Helm Chart v2.0.12
2. Generate manifests: helm template airbyte airbyte-v2/airbyte
3. Validate with kubeconform: kubeconform -strict manifest.yaml

### Expected Behavior

ConfigMap keys should be unique, with conditional logic determining which template generates the DATAPLANE configuration based on deployment architecture (hybrid vs data-plane vs control-plane).

Suggested Fix

Add conditional logic to prevent both templates from generating the same keys, likely based on global.cluster.type configuration:

#### In env-configmap.yaml

```yaml
{{- if ne (include "airbyte.common.cluster.type" .) "data-plane" }}
{{- include "airbyte.auth.bootstrap.configVars" . | nindent 2 }}
{{- end }}
```

**OR alternatively, make `workloadLauncher.dataPlane.configVars` conditional**

### Environment

- Chart Version: `2.0.12`
- Airbyte Version: `1.8.2`
- Validation Tool: `kubeconform v0.6.7`
- Kubernetes Version: `1.30+`

### Relevant log output

```shell
airbyte.yaml - ConfigMap airbyte-airbyte-env failed validation: error unmarshalling resource: error converting YAML to JSON: yaml: unmarshal errors:
line 264: key "DATAPLANE_CLIENT_ID_SECRET_KEY" already set in map
line 266: key "DATAPLANE_CLIENT_SECRET_SECRET_KEY" already set in map
```

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.