AbsaOSS / AbsaOSS/enceladus

Run has duplicate field - unique ID

オープン
#1,102 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
Menas priority: medium refactoring under discussion
主要言語
Scala
スター
33
フォーク
16
PR マージ指標
30日以内にマージされた PR はありません

説明

## Background
For historical reasons the `Run`'s `uniqueId` field is duplicated in `controlMeasure.runUniqueId` as it represents the contents of the `_INFO` file. The `_INFO` file references the run's unique ID, so a `conformance` job can find the correct `standardization job` to continue from, as `standardization + conformance` together constitute a single run.

## Feature
We should reduce this noise and only keep one field.

## Proposed Solution [Optional]
Solution Ideas
1. We can remove the top level field `uniqueId` OR
1. We can not include the `controlMeasure.runUniqueId` from the `_INFO` file
1. ~~We might just use the `Spark application ID`~~
1. We should make this field non-optional
1. Maybe we can just have a separate file (e.g. `_MENAS_RUN`), written by `Standardization` and read by `Conformance` to link the two spark jobs that doesn't need to be propagated to publish.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。