AbsaOSS / AbsaOSS/spline-spark-agent

Add Databricks environment details to execution plan

Abierto
#409 10 comentarios 0 reacciones 0 asignados Ver en GitHub
dependency: Databricks feature
Lenguaje dominante
Scala
Estrellas
210
Forks
102
Merge medio
1 d 1 h
PR fusionados (30 d)
1

Descripción

@wajda @cerveada , after taking some inspiration from Openlineage installtion steps, I was able to do codeless spline installation on Databricks (please see attached document for detailed steps). What I am missing now is to get some additional details from Databricks cluster like notebook name, user etc.. is it possible to build this functionality in spline jar , so we don't need run extra code snippet?

This has been implemented in [openlineage ](https://github.com/OpenLineage/OpenLineage/blob/main/integration/spark/src/main/common/java/io/openlineage/spark/agent/facets/builder/DatabricksEnvironmentFacetBuilder.java) using a databricks environment facet builder. I hope this can be done in spline as well.

[Install_Spline_Codeless.docx](https://github.com/AbsaOSS/spline-spark-agent/files/8161698/Install_Spline_Codeless.docx)

[Shell_Scripts.zip](https://github.com/AbsaOSS/spline-spark-agent/files/8161717/Shell_Scripts.zip)

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.