envoyproxy / envoyproxy/gateway
extension manager: missing backendResource CRD/RBAC crashes the controller instead of degrading gracefully
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
## What happened
With `extensionManagers.backendResources` pointing at a custom CRD (e.g. `foo.v1.bar.com.br`), the controller adds a watch for that GVK. If the CRD is absent — or the controller's ServiceAccount lacks `list`/`watch` RBAC on it — the informer's initial LIST fails and the controller never becomes Ready:
```
Failed to watch ... awsbackends.igw.ifood.com.br is forbidden:
User "system:serviceaccount:...:envoy-gateway" cannot list resource "awsbackends" ...
Could not wait for Cache to sync ... timed out waiting for cache to be synced
for Kind *v1.GatewayClass (and every other watched kind)
unable to start provider ... failed to wait for gatewayapi caches to sync ... timed out
leader election lost
```
The manager exits and the pod crash-loops.
## Impact
A missing custom CRD (or a single missing RBAC rule) takes down the **entire** gateway controller for that GatewayClass — not just routes using the custom backend. All routes (custom and non-custom) stop being reconciled/served.
## Expected behavior
Degrade gracefully: log a warning and skip the watch for that backendResource (disabling only the extension manager for that GVK), while continuing to serve everything else. This matches the existing pattern:
```
ServiceImport CRD not found, skipping ServiceImport watch
```
## Suggested fix
Make `backendResources` watch best-effort — e.g. a `watchOnError: continue` (or `optional: true`) option on the extension manager, so a LIST/RBAC failure disables only the extension manager instead of failing `WaitForCacheSync`.
## Willing to contribute
I'd be happy to work on a PR for this fix. The pattern already exists in the codebase for ServiceImport (skip watch on missing CRD), so applying the same to `backendResources` should be a targeted change. Happy to discuss the design — whether it should be a new `optional`/`watchOnError` field, or just always best-effort like ServiceImport.
## Environment
- Envoy Gateway v1.8.0 (official `envoyproxy/gateway:v1.8.0` image)
- `extensionManagers[].backendResources = [{group: foo.bar.com.br, version: v1alpha1, kind: AWSBackend}]`
- `provider.kubernetes.deploy.type: GatewayNamespace`, `watch.type: Namespaces`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the extension manager's backendResources watch setup and the existing ServiceImport watch that skips a missing CRD. Follow how informer LIST errors reach WaitForCacheSync and the provider startup path. Done means a missing CRD or list/watch RBAC failure logs a warning, disables only that backendResource watch, and leaves the controller Ready for other resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100