envoyproxy / envoyproxy/envoy

Delay ADS stream until XDS cluster warming

Offen
#7,253 6 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
api/v3 bug help wanted
Vorherrschende Sprache
C++
Sterne
28.9k
Forks
5.6k
Ø Merge
1 T. 22 Std.
Gemergte PRs (30 T.)
430

Beschreibung

Currently the ADS stream is started immediately after static load completes:

https://github.com/envoyproxy/envoy/blob/master/source/common/upstream/cluster_manager_impl.cc#L283

When used with STRICT_DNS, this means that the gRPC stream keeps attempting to connect until DNS resolves, which can exacerbate startup delays due to DNS issues: the stream will continuously attempt to connect, building up the retry backoff counter. By the time DNS finally resolves, the next attempt might be scheduled several seconds into the future. As a result, a 5 second DNS delay (eg due to a timeout) might result in delaying startup by >10s.

While this slowdown isn't a major issue, it's caused short-ish DNS issues to cause significant deployment delays for us.

Conceptually the fix sounds like it would be to defer ADS stream start until the associated cluster warms (eg wait for first DNS resolution), but I'm not sure how easy it would be to express that with the current init helper.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.