AbsaOSS / AbsaOSS/enceladus

New version creation of a dataset do not carry over the properties

オープン
#1,698 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
bug priority: undecided
主要言語
Scala
スター
33
フォーク
16
PR マージ指標
30日以内にマージされた PR はありません

説明

## Describe the bug
Having a testing required properties key on my local Menas and having such a key-value pair in the properties of a dataset I want to run, I have discovered that if a new version of the dataset is created (e.g. by adding description, adding a conformance rule, ...) the properties set to the previous version do not carry over.

This, in turn, may prevent running standardization/conformance if the system has any required properties keys set.

## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Have a dataset name `ds1` in the latest version `7` that works for std/conf
2. Update your system to require a property `requiredKey1`:
`POST http://localhost:8080/menas/api/properties/datasets` with a payload
```json
{
"name": "requiredKey1",
"description": "myDesc",
"propertyType": {
"_t": "StringPropertyType",
"suggestedValue": "sortOfDefault"
},
"essentiality": {
"_t": "Mandatory"
}
}
```
3. This renders your dataset `ds1` unusable because it lacks the required key-value in its properties
4. Fix the dataset by providing the required key-value by
`PUT http://localhost:8080/menas/api/dataset/ds1/properties` with a payload
```json
{
"requiredKey1": "valueOfRequiredKey1"
}
```
resulting in version 8 of `ds1` that is usable for std/conf
5. Edit the dataset `ds1` by changing its description resulting in version 9.
6. Dataset `ds1` in version 9, now, has no properties set and is unusable for std/conf processing again (spark-submit will fail with this message)

## Expected behavior
It would be expected that the properties would carry over from v 8 to v 9.

## Comments
This is quite serious in cases where we expect that there may be any properties set in the system, until such time, the problem should not be encountered. That being said, it should be probably fixed sooner than later.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。