grpc / grpc/grpc-java

xDS ADS reconnection should not spam logs

Aperta
#8,886 2 commenti 1 reazione 0 assegnatari Vedi su GitHub
bug xds
Lingua principale
Java
Stelle
12.1k
Fork
4k
Merge medio
2g 17h
PR unite (30g)
37

Descrizione

As recently seen:

```
WARNING: [Channel<11>: (xds:///)] Failed to resolve name. status=Status{​​code=UNAVAILABLE, description=Closed by server, cause=null}
```

This is being logged from ManagedChannelImpl:
https://github.com/grpc/grpc-java/blob/7308d920346e2e1cae640ed1f7e4dfad1b032df8/core/src/main/java/io/grpc/internal/ManagedChannelImpl.java#L1891-L1893

I'm surprised that log statement exists as a WARNING. Seems it was introduced in the large ManagedChannelImpl2 rewrite (#2530).

There's two cases this WARNING could happen:
1. Before resolution has ever succeeded. There's no need for logging because the error will be communicated to RPCs. This is unlikely the case here just because "Closed by server" is probably the cycling of long-lived RPCs for load balancing
2. After initial resolution. This might deserve logs at some points because RPCs probably won't fail (gRPC will continue using previous data). In this case, xds client should probably squelch this "error" as it is quite normal. There could be some debate here as to whether the watchers would be notified, or whether it matters if they are notified, but overall this is quite ordinary. Any error here should be logged if re-creating the ADS stream fails (which is probably dependent on whether the stream received responses).

So that means we should make two changes (one to ManagedChannelImpl, one to xds).

I'd hope we'd never log WARNINGs like this, but it is a bit hard of a situation and I don't want us to let perfect be the enemy of good. At least for xDS, we should be able to avoid logging except when ADS stream creation fails.

I've seen #8773, but it looks incomplete, as it'd need to handle the "never received a response" case.

CC @erikjoh, @dapengzhang0, @YifeiZhuang, @sergiitk

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start at ManagedChannelImpl.java around the linked warning and inspect the xDS client handling discussed in the issue and #8773. Trace the cases before and after the first response, then verify that ordinary ADS cycling is quiet while ADS stream creation failures remain logged, including the never-received-a-response case.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
distributed-systems, networking
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.