grpc / grpc/grpc-java

Graceful shutdown within servlet container results in NoClassDefFoundError being thrown asynchronously

Offen
#9,510 12 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Java
Sterne
12.1k
Forks
4k
Ø Merge
2 T. 17 Std.
Gemergte PRs (30 T.)
37

Beschreibung

### What version of gRPC-Java are you using?

`io.grpc:grpc-netty-shaded:1.46.0`

### What is your environment?

macOS / Linux / JDK 15 / Jetty 10

### What did you expect to see?

No exception being thrown after a graceful shutdown. Graceful shutdown happens during undeployment of web application.

### What did you see instead?

The log indicates that graceful shutdown has succeeded and consequently the web application has been undeployed. However, roughly a second later Netty throws `NoClassDefFoundException` seen below. Seems like `channel.awaitTermination()` returns too early.

```
06.09.2022 14:07:47.465 | grpc-shared-destroyer-0 | ERROR io.grpc.internal.LogExceptionRunnable
Exception while executing runnable io.grpc.internal.SharedResourceHolder$2@58725740
java.lang.ClassNotFoundException: io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise$1
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:606)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:511)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 17 common frames omitted
Wrapped by: java.lang.NoClassDefFoundError: io/grpc/netty/shaded/io/netty/util/concurrent/DefaultPromise$1
at io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:499)
at io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
at io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
at io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.ensureThreadStarted(SingleThreadEventExecutor.java:961)
at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.shutdownGracefully(SingleThreadEventExecutor.java:660)
at io.grpc.netty.shaded.io.netty.util.concurrent.MultithreadEventExecutorGroup.shutdownGracefully(MultithreadEventExecutorGroup.java:163)
at io.grpc.netty.shaded.io.grpc.netty.Utils$DefaultEventLoopGroupResource.close(Utils.java:439)
at io.grpc.netty.shaded.io.grpc.netty.Utils$DefaultEventLoopGroupResource.close(Utils.java:402)
at io.grpc.internal.SharedResourceHolder$2.run(SharedResourceHolder.java:139)
at io.grpc.internal.LogExceptionRunnable.run(LogExceptionRunnable.java:43)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
```

### Steps to reproduce the bug

Use a servlet container, e.g. Jetty, and register a `ServletContextListener` that starts and stops a GRPC client to call an external service. When undeploying the web application, the exception is sometimes thrown.

```java
public final class MyGrpcServiceClient {

private ManagedChannel channel;

public void start() {
checkState(channel == null, "already started");
chanel = ManagedChannelBuilder.forTarget("dns:///.....").usePlaintext().build();
}

public void stop() {
checkState(channel != null, "already stopped")
channel.shutdown();
try {
if (!channel.awaitTermination(1, TimeUnit.MINUTES)) {
// log warning that graceful shutdown has taken too long
}
} catch (InterruptedException e) {
// log that graceful shutdown was interrupted
} finally {
channel = null;
}
}

// ... factory methods for returning new GRPC service stubs using above channel ...
}

public final class MyGrpcServiceClientSCL implements ServletContextListener {

private MyGrpcServiceClient client = new MyGrpcServiceClient();

@Override
public void contextInitialized(ServletContextEvent e) {
try {
client.start();
// log successful start of service client
} catch (Exception e) {
throw new Error(e);
}
}

@Override
public void contextDestroyed(ServletContextEvent e) {
try {
client.stop();
// log successful shutdown of service client
} catch (Exception e) {
// log unexpected exceptions
}
}

}

```

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit der Reproduktion von ServletContextListener/MyGrpcServiceClient und untersuche den im Trace genannten Shutdown-Pfad, insbesondere SharedResourceHolder und Utils$DefaultEventLoopGroupResource. Vergleiche channel.awaitTermination mit der späteren asynchronen Bereinigung während des Undeployments des Servlets. Erledigt ist die Aufgabe, wenn der ordnungsgemäße Shutdown ohne einen verzögerten NoClassDefFoundError oder eine verwandte Exception abgeschlossen wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
java
Bereich
api, backend
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.