AbsaOSS / AbsaOSS/spline-spark-agent
Add Databricks environment details to execution plan
- Ngôn ngữ chính
- Scala
- Star
- 210
- Fork
- 102
- Merge trung bình
- 1 ngày 1 giờ
- Pull request đã merge (30 ngày)
- 1
Mô tả
@wajda @cerveada , after taking some inspiration from Openlineage installtion steps, I was able to do codeless spline installation on Databricks (please see attached document for detailed steps). What I am missing now is to get some additional details from Databricks cluster like notebook name, user etc.. is it possible to build this functionality in spline jar , so we don't need run extra code snippet?
This has been implemented in [openlineage ](https://github.com/OpenLineage/OpenLineage/blob/main/integration/spark/src/main/common/java/io/openlineage/spark/agent/facets/builder/DatabricksEnvironmentFacetBuilder.java) using a databricks environment facet builder. I hope this can be done in spline as well.
[Install_Spline_Codeless.docx](https://github.com/AbsaOSS/spline-spark-agent/files/8161698/Install_Spline_Codeless.docx)
[Shell_Scripts.zip](https://github.com/AbsaOSS/spline-spark-agent/files/8161717/Shell_Scripts.zip)
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.