dotnet / dotnet/spark

Getting [Exception] [JvmBridge] Connection refused 127.0.0.1:5567 when running example project

Open
#1,110 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

Problem encountered on https://dotnet.microsoft.com/en-us/learn/data/spark-tutorial/run
Operating System: linux

Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

I am following the instructions of setting up a test app using dotnet Spark when I try to run it. The error is:

[Exception] [JvmBridge] Connection refused 127.0.0.1:5567
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at Microsoft.Spark.Network.DefaultSocketWrapper.Connect(IPAddress remoteaddr, Int32 port, String secret)
at Microsoft.Spark.Interop.Ipc.JvmBridge.GetConnection()
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
Unhandled exception. System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (61): Connection refused 127.0.0.1:5567
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
at Microsoft.Spark.Network.DefaultSocketWrapper.Connect(IPAddress remoteaddr, Int32 port, String secret)
at Microsoft.Spark.Interop.Ipc.JvmBridge.GetConnection()
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
at Microsoft.Spark.Interop.Ipc.JvmBridge.CallStaticJavaMethod(String className, String methodName, Object[] args)
at Microsoft.Spark.Sql.Builder..ctor()
at Microsoft.Spark.Sql.SparkSession.Builder()
at MySparkApp.Program.Main(String[] args) in /Users/kurshu/source/DataManagementFactory/mySparkApp/Program.cs:line 10

I am following the instruction found here:

I am running on a MAC (OS: Monterey)

https://dotnet.microsoft.com/en-us/learn/data/spark-tutorial/run

Thanks for your help!

Contributor guide

Open the contributing guide

Research direction

Start with the run tutorial at https://dotnet.microsoft.com/en-us/learn/data/spark-tutorial/run and the example entry point in Program.cs at line 10. Trace the JvmBridge connection shown in the error and verify the setup that should provide 127.0.0.1:5567; done means the example starts without the connection-refused exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
data-engineering, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.