AbsaOSS / AbsaOSS/spline-spark-agent

Databricks - Support of Delta live tables

未关闭
#813 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
help wanted
主要语言
Scala
星标
210
派生
102
平均合并
1 天 1 小时
30 天内合并 PR
1

描述

I have a case of using delta live tables that runs SQL notebooks.

I cannot add cell of python
`sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession)`
since DLT doesn't support magic like `%python`
So I created a separate notebook of python - but still fails - although the spline Jar is installed on the cluster
I think that because DLT manages dynamically the clusters - I think that all the installation of jars on the cluster take no effect.

## Question
Does someone has experience using spline in Databricks in DLT?

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。