envoyproxy / envoyproxy/gateway
Flapping: TLS error: Secret is not supplied by SDS
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
*Description*:
I am testing envoy gateway in a k3s multinode setup (v1.32.6+k3s1) with two envoy gateways (internal and external running gateway:v1.4.2 and envoyproxy/envoy:distroless-v1.34.1).
The purpose of the two gateways is to implement internal and external access with different SecurityPolicy's objects applied. Therefore I have 2 HTTPRoutes which only differ in the gateway those are attached (parentRefs.name) and I have two different SecurityPolicys. The HTTPRoutes route towards the same https upstream/backend k8s service. I have one BackendTLSPolicy in place which is attached to the https backend service showing this validation (a trust to the root ca and the SAN/hostname):
```
validation:
hostname: somthing.fqdn
caCertificateRefs:
- name: clusterissuer-root-ca01
group: ''
kind: ConfigMap
```
The config map equals this setup (and contains only the root ca certificate, the signer of the upstream/backend certificate):
kubectl create configmap clusterissuer-root-ca01 --from-file=ca.crt -n namespace
The setup works fine until it begins to flap (without any change to the HTTPRoute, BackendTLSPolicy nor the clusterissuer-root-ca01 configmap or the services etc.) and both https route return with http status code 503 and error:
* upstream connect error or disconnect/reset before headers. reset reason: remote connection failure,
transport failure reason: TLS error: Secret is not supplied by SDS
If I check the generated envoy configuration I can see that the secret (ca root trust) was moved to the dynamic_warming_secrets and remains with "version_info": "uninitialized" sometimes for approx 5min but sometimes even for hours.
```
"dynamic_warming_secrets": [
{
"name": "internal/namespace-ca",
"version_info": "uninitialized",
"last_updated": "2025-07-28T16:51:15.106Z",
"secret": {
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
"name": "internal/namespace-ca"
}
}
]
```
and not as expected:
```
"dynamic_active_secrets": [
...
{
"name": "internal/namespace-ca",
"version_info": "086a249165bd6d0c6883a81c8fef598a5265a3af37bc657faec1c8eaa067cbbb",
"last_updated": "2025-06-18T18:02:36.577Z",
"secret": {
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
"name": "internal/namespace-ca",
"validation_context": {
"trusted_ca": {
"inline_bytes": "omitted_base64(pem of ca root cert)"
}
}
}
}
]
```
*Repro steps*:
Unkown so far, but the flapping occurs daily.
*Environment*:
* v1.32.6+k3s1
* gateway:v1.4.2
* envoyproxy/envoy:distroless-v1.34.1
*Logs*:
debug mode of gateway show no errors
How may I debug further or what component will take care to complete the dynamic_warming_secrets - is it gateway or envoyproxy?
/martin
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.