envoyproxy / envoyproxy/gateway
Endpoint fast path: propagate endpoint updates without waiting for full translation (EDS-only pushes)
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
## Description
In large clusters, endpoint updates are delayed by however long a full translation takes, because endpoints only reach Envoy by riding along in a complete rebuild of everything. When translations are slow, Envoy routes to stale endpoints for the entire build duration, which can cause availability loss.
## Current behavior
Every EndpointSlice event takes the full pipeline:
1. Provider: full reconcile rebuilds the entire resource tree for the GatewayClass.
2. Gateway API runner: full translation of all resources to IR.
3. xDS runner: full translation of the IR to all xDS resource types, then a new snapshot.
While a build is in flight, watchable coalescing merges incoming updates into the next build — so endpoints continuously lag by roughly one full build duration.
At large scale (tens of thousands of routes, many EnvoyPatchPolicy resources with expensive JSON patches), a single full translation can take multiple seconds to tens of seconds. Under heavy churn, builds run back-to-back, so endpoint staleness is sustained rather than episodic.
## Why this needs decoupling, not just optimization
The main reason is update frequency: in a large deployment with constant pod churn (rollouts, autoscaling, node drains), endpoint updates are multiple orders of magnitude more frequent than any other input — endpoints change at pod timescale while routes and policies change at human timescale. Today the highest-frequency, cheapest-to-compute signal flows through the lowest-frequency, most expensive machinery: every endpoint change pays the cost of a full translation, and under sustained churn that cost is paid continuously.
Decoupling also isolates faults: the main loop can be degraded by expensive or invalid configuration — very large route counts, costly EnvoyPatchPolicy JSON patches, broken references re-resolved every cycle. In a multi-tenant setup, one tenant's config delays endpoint delivery for every tenant sharing the control plane. Optimizing translation shrinks the staleness window, but only decoupling removes the failure class.
## Proposal
An opt-in **endpoint fast path**: EndpointSlice changes are published on a dedicated channel and translated into updated `ClusterLoadAssignment`s using a **cached per-cluster endpoint context** from the last successful full build (destination shape, health check config, zone/locality settings), then patched into the current snapshot with only the EDS version bumped. Semantics are **best-effort**: fresh endpoints against last-known-good config; full builds remain authoritative and re-apply the latest endpoint state on publish.
Cases where an endpoint change affects config-owned behavior escalate to the full path (e.g., zero↔non-zero endpoint transitions, clusters targeted by EnvoyPatchPolicy CLA patches or extension-server endpoint hooks).
**Prior art:** Istio's incremental (EDS-only) push — endpoint updates bypass `PushContext` recomputation, regenerate only affected CLAs, and escalate to a full push when cluster-level config is affected (`DiscoveryServer.EDSUpdate`, `EndpointIndex.UpdateServiceEndpoints`).
**Relationship to update debouncing #9803:** complementary — debouncing bounds push *frequency*; this bounds endpoint *staleness*. The fast-path channel would use the same debounce machinery with shorter intervals.
## Alternatives considered
- **Optimizing translation only** — worthwhile independently, but leaves endpoint freshness coupled to worst-case config cost.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.