apache / apache/beam

Flink 1.12 job server error: Trying to access closed classloader.

Open
#20,783 0 comments 0 reactions 0 assignees View on GitHub
bug flink P3 portability runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

When running multiple jobs in local mode on the Flink 1.12 job server, pipeline construction for subsequent jobs fails with the error below. Classloader leakage is being detected by the wrapper classloader (see FLINK-16245).

The classloader is probably being leaked by GRPC threads that we can't shut down cleanly.

RuntimeError: java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'. at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:161) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:168) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.beam.sdk.options.PipelineOptionsFactory.findCallersClassName(PipelineOptionsFactory.java:418) at org.apache.beam.sdk.options.PipelineOptionsFactory.access$100(PipelineOptionsFactory.java:122) at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.(PipelineOptionsFactory.java:212) at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.(PipelineOptionsFactory.java:208) at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.(PipelineOptionsFactory.java:199) at org.apache.beam.sdk.options.PipelineOptionsFactory.create(PipelineOptionsFactory.java:131) at org.apache.beam.sdk.Pipeline.create(Pipeline.java:146) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:394) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:491) at org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:232) at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172) at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331) at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817) at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at org.apache.beam.vendor.grpc.v1p26p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Commit/1707/testReport/junit/apache_beam.runners.portability.flink_runner_test/FlinkRunnerTestStreaming/test_external_transform/

Imported from Jira [BEAM-11570](https://issues.apache.org/jira/browse/BEAM-11570). Original Jira may contain additional context.
Reported by: ibzib.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.