AbsaOSS / AbsaOSS/spline-spark-agent

Databricks - Support of Delta live tables

Aperta
#813 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
help wanted
Lingua principale
Scala
Stelle
210
Fork
102
Merge medio
1g 1h
PR unite (30g)
1

Descrizione

I have a case of using delta live tables that runs SQL notebooks.

I cannot add cell of python
`sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession)`
since DLT doesn't support magic like `%python`
So I created a separate notebook of python - but still fails - although the spline Jar is installed on the cluster
I think that because DLT manages dynamically the clusters - I think that all the installation of jars on the cluster take no effect.

## Question
Does someone has experience using spline in Databricks in DLT?

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.