Manual mTLS Certificate Update Gets Overwritten Due to Missing ConfigMap Update
- Dominant language
- SCSS
- Stars
- 1k
- Forks
- 794
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 7
Description
---
### **Issue**
While following the [[official Dapr documentation](https://docs.dapr.io/operations/security/mtls/#custom-certificates-bring-your-own)](https://docs.dapr.io/operations/security/mtls/#custom-certificates-bring-your-own) for configuring custom mTLS certificates, there are three methods listed:
1. Using the Dapr CLI
2. Using the Helm upgrade
3. Manually updating the `dapr-trust-bundle` secret
Currently, both the **Dapr CLI** and **Helm** methods **fail** under certain conditions:
#### **Dapr CLI issue:**
If the Dapr control plane was installed via Azure CLI (`az aks enable-addons --addons dapr --name --resource-group `), it installs a chart version such as `1.15.3-msft.2` which is not available in the official [[Dapr Helm chart repo](https://dapr.github.io/helm-charts)](https://dapr.github.io/helm-charts). Running the CLI renewal fails with:
```
certificate rotation failed: chart "dapr" version "1.15.3-msft.2" not found in https://dapr.github.io/helm-charts repository
```
#### **Helm upgrade issue:**
Attempting to use `helm upgrade` fails due to **mismatched properties between the Microsoft-specific chart version and the public Dapr chart repo**. The `values.yaml` structure and expected parameters differ, resulting in errors during upgrade (e.g., unknown fields, incompatible chart values, etc.).
Due to these issues, the only viable method is **manual secret update**. But even this has a caveat...
#### **Secret rollback issue:**
After manually updating the `dapr-trust-bundle` Secret with custom certificates (`issuer.crt`, `issuer.key`, `ca.crt`), the changes are **automatically reverted** shortly after. This behavior occurs because the Dapr Helm chart also provisions a dapr-trust-bundle ConfigMap, which contains the root certificate (ca.crt) and is referenced by the Sentry deployment.
If the ConfigMap is not updated to match the new certificate values, it causes the system to regenerate and reapply the original Secret values - effectively overriding any manual updates.
This behavior is defined in the Dapr Helm chart template used for deploying Dapr Sentry:
🔗 [dapr_sentry_deployment.yaml](https://github.com/dapr/dapr/blob/80e00a6847f26875a11d6a0eb1de4ac98c423bde/charts/dapr/charts/dapr_sentry/templates/dapr_sentry_deployment.yaml#L18)
---
### **URL of the docs**
- https://docs.dapr.io/operations/security/mtls/#custom-certificates-bring-your-own
- https://learn.microsoft.com/en-us/azure/aks/dapr-settings?tabs=cli#generate-a-new-dapr-generated-self-signed-certificate
- https://learn.microsoft.com/en-us/azure/aks/dapr-settings?tabs=cli#manage-your-own-user-supplied-x509-certificates
---
### **Expected content**
The documentation should clearly state that:
- If using the **manual update method**, you must **first update the `dapr-trust-bundle` ConfigMap**, followed by the Secret. Otherwise, the custom values will be overwritten.
- The **Dapr CLI** may not work if the installed version is Microsoft-specific (`*-msft.*`) and doesn't exist in the official Helm repo.
- The **Helm upgrade path fails** due to mismatch in chart structure and parameters between the Microsoft-maintained and official Dapr Helm charts.
---
### **Screenshots**
- **CLI Failure:**

- **Helm chart version mismatch:**

---
### **Additional context**
A related issue was also raised here:
🔗 [Azure AKS Docs GitHub Issue](https://github.com/MicrosoftDocs/azure-aks-docs/issues/211)
The current workaround is:
1. **First update the `dapr-trust-bundle` ConfigMap** with the new `ca.crt` value.
2. **Then update the Secret** with the base64-encoded versions of the custom `issuer.crt`, `issuer.key`, and `ca.crt`.
This critical step of updating the ConfigMap is not currently documented in the official Dapr guide and should be added to prevent confusion and failed manual certificate setups.
---
Contributor guide
Assessment
This issue has not been assessed yet.