AbsaOSS / AbsaOSS/spline-spark-agent

Add Databricks environment details to execution plan

未关闭
#409 10 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
dependency: Databricks feature
主要语言
Scala
星标
210
派生
102
平均合并
1 天 1 小时
30 天内合并 PR
1

描述

@wajda @cerveada , after taking some inspiration from Openlineage installtion steps, I was able to do codeless spline installation on Databricks (please see attached document for detailed steps). What I am missing now is to get some additional details from Databricks cluster like notebook name, user etc.. is it possible to build this functionality in spline jar , so we don't need run extra code snippet?

This has been implemented in [openlineage ](https://github.com/OpenLineage/OpenLineage/blob/main/integration/spark/src/main/common/java/io/openlineage/spark/agent/facets/builder/DatabricksEnvironmentFacetBuilder.java) using a databricks environment facet builder. I hope this can be done in spline as well.

[Install_Spline_Codeless.docx](https://github.com/AbsaOSS/spline-spark-agent/files/8161698/Install_Spline_Codeless.docx)

[Shell_Scripts.zip](https://github.com/AbsaOSS/spline-spark-agent/files/8161717/Shell_Scripts.zip)

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。