envoyproxy / envoyproxy/envoy

source address bind errors fail all upstream retries

Offen
#9,595 13 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @jrajahalme Auf GitHub ansehen
area/connection enhancement help wanted
Vorherrschende Sprache
C++
Sterne
28.9k
Forks
5.6k
Ø Merge
1 T. 22 Std.
Gemergte PRs (30 T.)
430

Beschreibung

When upstream connections are used with a specified source address/port, and the upstream request fails, e.g., due to a timeout, it may happen that all retry attempts fail on a bind error. This is due to the new connections trying to bind to the same source address, while the address/port is apparently still in use even though the logs indicate the deferred deletion has been done before the new bind attempt.

Example logs:
```
< C9 is bound to the local address 10.10.0.14:56122 >
[C9] connecting to 104.31.77.248:443
[C9] connected
... < about 30 secs later > ...
[C9] ssl read 1031 bytes into 1 slices
[C9] parsing 1031 bytes
[C9] headers complete
[C9] completed header: ...
[C8][S1906562538001471449] upstream headers complete: end_stream=false
[C8][S1906562538001471449] performing retry
[C8][S1906562538001471449] resetting pool request
[C9] request reset
item added to deferred deletion list (size=1)
[C9] SSL shutdown: rc=0
[C9] disconnect. resetting 0 pending requests
[C9] client disconnected, failure reason:
item added to deferred deletion list (size=2)
item added to deferred deletion list (size=3)
[C9] message complete
[C9] message complete
[C9] parsed 1031 bytes
clearing deferred deletion list (size=3)
creating a new connection
Bind failure. Failed to bind to 10.10.0.14:56122: Address already in use
[C10] connecting
[C10] connecting to 104.31.77.248:443
[C10] raising bind error
[C10] closing socket: 1
[C10] purge pending, failure reason:
[C8][S1906562538001471449] upstream reset: reset reason connection failure
[C8][S1906562538001471449] performing retry
item added to deferred deletion list (size=1)
clearing deferred deletion list (size=1)
creating a new connection
Bind failure. Failed to bind to 10.10.0.14:56122: Address already in use
[C11] connecting
[C11] connecting to 104.31.77.248:443
[C11] raising bind error
[C11] closing socket: 1
[C8][S1906562538001471449] upstream reset: reset reason connection failure
[C8][S1906562538001471449] performing retry
creating a new connection
Bind failure. Failed to bind to 10.10.0.14:56122: Address already in use
[C12] connecting
[C12] connecting to 104.31.77.248:443
[C12] raising bind error
[C12] closing socket: 1
[C8][S1906562538001471449] upstream reset: reset reason connection failure
[C8][S1906562538001471449] Sending local reply with details upstream_reset_before_response_started{connection failure}
```

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.