dotnet / dotnet/spark

[BUG]: JvmBridge registering CLR garbage collector threads with JvmThreadPoolGC

Open
#1,002 3 comments 0 reactions 0 assignees View on GitHub
bug good first issue
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Describe the bug**
When the finalizer is called for CLR [JvmObjectId](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/csharp/Microsoft.Spark/Interop/Ipc/JvmObjectReference.cs#L33) objects, it calls the `rm` DotnetBackend method and this calls goes through the `JvmBridge` class. Because the `rm` call goes through [JvmBridge.CallJavaMethod](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/csharp/Microsoft.Spark/Interop/Ipc/JvmBridge.cs#L183) code path, it gets registered with [JvmThreadPoolGC.TryAddThread](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/csharp/Microsoft.Spark/Interop/Ipc/JvmThreadPoolGC.cs#L76).

Although this GC thread gets registered on the CLR side by the finalizer call, when the `rm` is handled by [DotnetBackendHandler](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/scala/microsoft-spark-2-4/src/main/scala/org/apache/spark/api/dotnet/DotnetBackendHandler.scala#L56), it does not run in a [org.apache.spark.api.dotnet.ThreadPool](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/scala/microsoft-spark-2-4/src/main/scala/org/apache/spark/api/dotnet/DotnetBackendHandler.scala#L104). On the CLR side there is a timer associated with [JvmThreadPoolGC](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/csharp/Microsoft.Spark/Interop/Ipc/JvmThreadPoolGC.cs#L113) and when it runs, it attempts to call `rmThread` on any non-active threads. Because the finalizer's `rm` JVM call did not run in a org.apache.spark.api.dotnet.ThreadPool, the Java [DotnetBackendHandler "rmThread"](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/scala/microsoft-spark-2-4/src/main/scala/org/apache/spark/api/dotnet/DotnetBackendHandler.scala#L80) call will log an error and the CLR [JvmThreadPoolGC](https://github.com/dotnet/spark/blob/b0a3068bd8df2a5e8d1754cf2f1f50d38d37f57b/src/csharp/Microsoft.Spark/Interop/Ipc/JvmThreadPoolGC.cs#L120) will log a warning.

Although this won't cause applications to fail, it makes extra calls to the DotnetBackend as well as produce warning and error log lines which is not a good experience.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.