envoyproxy / envoyproxy/gateway

Envoy OIDC SecurityPolicy fails on startup and doesn't recover

Open
#6,123 6 comments 2 reactions 0 assignees View on GitHub
kind/bug stale
Dominant language
Go
Stars
3k
Forks
864
Avg merge
2d 2h
Merged PRs (30d)
140

Description

Hello,

*Description*:

I am using an oidc auth SecurityPolicy with an issuer that is served from the same envoy gateway. Sometimes, at startup, it can happen that the issuer URL is not yet started up when the SecurityPolicy builds its configuration so it fails and does not recover. Looking at the documentation, I do not see how to configure it to retry. For me a good enough fix would be if it could recover from this error after some period of time.

*Repro steps*:

```
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
name: keycloak-auth-fqdn
namespace: prometheus
spec:
endpoints:
- fqdn:
hostname: auth.xxxx.com
port: 443
type: Endpoints
status:
conditions:
- lastTransitionTime: "2025-05-16T05:22:18Z"
message: The Backend was accepted
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
name: prometheus-oidc
namespace: prometheus
spec:
oidc:
clientID: envoy-gateway
clientSecret:
group: ""
kind: Secret
name: envoy-oidc
logoutPath: /logout
provider:
backendRefs:
- group: gateway.envoyproxy.io
kind: Backend
name: keycloak-auth-fqdn
port: 443
issuer: https://auth.xxxx.com/realms/master
redirectURL: https://prometheus.xxxx.com/oauth2/callback
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: prometheus
status:
ancestors:
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: eg
namespace: envoy-gateway-system
conditions:
- lastTransitionTime: "2025-05-20T13:46:53Z"
message: 'OIDC: error fetching endpoints from issuer: EOF.'
observedGeneration: 2
reason: Invalid
status: "False"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
```

*Environment*:

Envoy Gateway: v1.4.0
Kubernetes: v1.32.3+k3s1

*Logs*:

```
{":authority":"prometheus.xxxx.com","bytes_received":0,"bytes_sent":0,"connection_termination_details":null,"downstream_local_address":"10.42.0.25:10443","downstream_remote_address":"192.168.1.11:54492","duration":0,"method":"GET","protocol":"HTTP/2","requested_server_name":"prometheus.xxxx.com","response_code":500,"response_code_details":"direct_response","response_flags":"-","route_name":"httproute/prometheus/prometheus/rule/0/match/0/prometheus_xxxx_com","start_time":"2025-05-20T14:14:11.750Z","upstream_cluster":null,"upstream_host":null,"upstream_local_address":null,"upstream_transport_failure_reason":null,"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0","x-envoy-origin-path":"/query","x-envoy-upstream-service-time":null,"x-forwarded-for":"192.168.1.11","x-request-id":"09b28fc9-e32b-4c18-866a-ae871263e45f"}
```

Thanks,
Tiberiu

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.