envoyproxy / envoyproxy/gateway

Invalid tracing backendRef degrades Gateway, causing loss of load balancer address

Open
#9,229 6 comments 5 reactions 1 assignee Claimed by @vishwas-bm View on GitHub
kind/enhancement stale
Dominant language
Go
Stars
3k
Forks
864
Avg merge
2d 2h
Merged PRs (30d)
140

Description

*Description*:
After configuring an invalid tracing backendRef for an Envoy Gateway proxy, the Gateway was marked not accepted with `reason: InvalidParameters`. As a side effect, the Kubernetes `Service` backing the Gateway lost its load balancer address.

This caused `external-dns` to observe the load balancer address removal and update DNS records, resulting in an outage for traffic depending on those records.

Expected behavior: a bad tracing backend reference should surface as a configuration/status error, but it should not cause the Gateway resource to become degraded resulting in its load balancer address being removed or otherwise disrupt unrelated data-plane availability.

*Environment*:
- Envoy Gateway: 1.8.1

*Repro steps*:
1. Deploy Envoy Gateway with tracing enabled.
2. Configure tracing with a bad backend reference, for example pointing to a service that does not exist:

snippit from `envoyproxy` resource:
```yaml
tracing:
provider:
backendRefs:
- group: ""
kind: Service
name: alloy-receiver
namespace: grafana-alloy
port: 8126
weight: 1
port: 4317
type: Datadog
samplingRate: 5
```

3. Apply the configuration to a cluster where the Gateway has an external load balancer address.
4. Observe the Gateway `Accepted` condition become `False` with `InvalidParamaters`.
5. Observe the Gateway Service losing its `status.loadbalance.ingress` address.
6. If `external-dns` is installed, observe DNS records being updated/removed based on the missing load balancer address.

```yaml
status:
conditions:
- lastTransitionTime: "2026-06-15T23:48:33Z"
message: 'Invalid tracing backendRefs in the referenced EnvoyProxy: service grafana-alloy/grafana-alloy
not found'
observedGeneration: 6
reason: InvalidParameters
status: "False"
type: Accepted
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2026-06-15T23:48:33Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 6
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2026-06-15T23:48:33Z"
message: Listener has been successfully translated
observedGeneration: 6
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2026-06-15T23:48:33Z"
message: Listener references have been resolved
observedGeneration: 6
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.