envoyproxy / envoyproxy/gateway
sessionPersistence is not working when using Backend mTLS with a Service FQDN
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
*Description*:
When an HTTPRoute uses `sessionPersistence` with a direct Kubernetes Service backend, Envoy Gateway routes through EndpointSlice pod endpoints. The persistence cookie is tied to the selected pod endpoint, for example `pod_ip:port`, and repeated requests stay on the same pod.
When the same Service is referenced through a `gateway.envoyproxy.io/Backend` using a Kubernetes Service DNS name, for example `tls-backend.default.svc.cluster.local`, Envoy Gateway treats the Backend endpoint as a normal FQDN endpoint. In practice this produces a cookie/backend identity based on the Service DNS/ClusterIP path, for example `cluster_ip:port`, and traffic can land on different pods across requests.
This makes session persistence behave differently for two configurations that both logically target the same Service:
1. Plain backend TLS:
- HTTPRoute backendRef points directly to `Service/tls-backend`.
- `BackendTLSPolicy` configures backend TLS.
- Session persistence uses pod endpoints.
2. Backend mTLS:
- HTTPRoute backendRef points to `Backend/tls-backend-client-cert`.
- `Backend.spec.endpoints[].fqdn.hostname` is `tls-backend.default.svc.cluster.local`.
- Backend configures client certificate and CA references.
- Session persistence uses the Service DNS/ClusterIP path and is not sticky to a pod.
*Repro Steps*:
1. Install Envoy Gateway with the Backend API enabled.
2. Deploy a backend Service with at least two ready pods. The Service should be reachable as `tls-backend.default.svc.cluster.local` and expose port `443`.
3. Configure HTTPRoute session persistence against the Service directly:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: sticky-service
namespace: default
spec:
parentRefs:
- name: eg
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: tls-backend
port: 443
sessionPersistence:
sessionName: Session-A
type: Cookie
absoluteTimeout: 10s
cookieConfig:
lifetimeType: Permanent
```
4. Configure backend TLS for the Service with `BackendTLSPolicy` as usual.
5. Send a request, capture the persistence cookie, then send several more requests with that cookie.
Expected and observed for this case:
- Requests stay on the same pod.
- The persistence cookie/backend identity corresponds to a pod endpoint, for example `pod_ip:port`.
6. Replace the direct Service backendRef with a `Backend` backendRef and define the Backend with an FQDN endpoint pointing at the same Service DNS name:
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
name: tls-backend-client-cert
namespace: default
spec:
endpoints:
- fqdn:
hostname: tls-backend.default.svc.cluster.local
port: 443
tls:
clientCertificateRef:
kind: Secret
name: example-client-cert
caCertificateRefs:
- group: ""
kind: ConfigMap
name: example-ca
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: sticky-backend
namespace: default
spec:
parentRefs:
- name: eg
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- group: gateway.envoyproxy.io
kind: Backend
name: tls-backend-client-cert
sessionPersistence:
sessionName: Session-A
type: Cookie
absoluteTimeout: 10s
cookieConfig:
lifetimeType: Permanent
```
7. Send the same request sequence with the persistence cookie.
Actual result:
- The cookie/backend identity is based on the Service DNS/ClusterIP path, for example `cluster_ip:port`.
- Requests can hit different backend pods even when the same persistence cookie is sent.
Expected result:
- Because the Backend FQDN names an in-cluster Service, Envoy Gateway should produce endpoint-routed configuration using the Service's EndpointSlices, preserving mTLS settings from the Backend.
- The sticky session behavior should match the direct Service + BackendTLSPolicy case.
*Environment*:
- Envoy Gateway version: `v1.7.1`
- Envoy version: `v1.37.1`
- Kubernetes version/distribution: `v1.34.5`
- Backend API enabled: `yes`
*Logs*:
No specific logs are generated.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.