AbsaOSS / AbsaOSS/spline-spark-agent

Databricks - Support of Delta live tables

Ouverte
#813 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
help wanted
Langage dominant
Scala
Étoiles
210
Forks
102
Merge moyen
1 j 1 h
PR mergées (30 j)
1

Description

I have a case of using delta live tables that runs SQL notebooks.

I cannot add cell of python
`sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession)`
since DLT doesn't support magic like `%python`
So I created a separate notebook of python - but still fails - although the spline Jar is installed on the cluster
I think that because DLT manages dynamically the clusters - I think that all the installation of jars on the cluster take no effect.

## Question
Does someone has experience using spline in Databricks in DLT?

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.