Getting 'Atum already initialized' error when trying to run Std+Conf multiple times
- 主要言語
- Scala
- スター
- 33
- フォーク
- 16
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
## Describe the bug
The error occurs when Standardization+Conformance jobs from a single Spark session. This happenned when running from Pramen since Pramen runs multiple Standardization+Conformance jobs in a single Spark Session.
## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Configure Pramen to run EnceladusSink for multiple tables in a pipeline.
2. Run the pipeline.
4. The error will occur saying "Control Measurements Tracking already initialized"
## Expected behavior
Allow running multiple Standardization+Conformance jobs multiple times from teh same Spask session.
## Screenshots
N/A
## Desktop (please complete the following information):
This was happening both in UAT and in PROD.
## Additional context
There is already logic in Enceladus that explicitly disables control metrics when conformance jobs runs after standardization. So single Standardization, Conformance, and Standardization+Conformance jobs run fine. The issue happens only if both Standardization and Conformance run multiple times.
コントリビューションガイド
評価
この issue はまだ評価されていません。