dotnet / dotnet/spark

Accessing azure databricks table using spark from azure function

Open
#1,022 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

I am trying to access azure databricks table. I have created a time trigger azure function which should connect to the databricks. I have used the jdbc url, personal access token available in ADB

when I run the azure function, I get an error
System.Private.CoreLib: Exception while executing function: GetMetaRegistryData. System.Net.Sockets: No connection could be made because the target machine actively refused it. <>:5567.

SparkSession spark = SparkSession
.Builder()
.AppName("Accessing ADB table from Azure fn")
.GetOrCreate();

DataFrame jdbcDf = spark.Read()
.Format("jdbc")
.Options(
new Dictionary
{
{"url", "jdbc:spark://adb-instanceURL/databasename;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/;UID=token;PWD=peronal acess token"},
{"dbtable", "mytablename"}
})
.Load();

jdbcDf.Show();
spark.Stop();

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.