AbsaOSS / AbsaOSS/enceladus

New version creation of a dataset do not carry over the properties

Đang mở
#1,698 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug priority: undecided
Ngôn ngữ chính
Scala
Star
33
Fork
16
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

## Describe the bug
Having a testing required properties key on my local Menas and having such a key-value pair in the properties of a dataset I want to run, I have discovered that if a new version of the dataset is created (e.g. by adding description, adding a conformance rule, ...) the properties set to the previous version do not carry over.

This, in turn, may prevent running standardization/conformance if the system has any required properties keys set.

## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Have a dataset name `ds1` in the latest version `7` that works for std/conf
2. Update your system to require a property `requiredKey1`:
`POST http://localhost:8080/menas/api/properties/datasets` with a payload
```json
{
"name": "requiredKey1",
"description": "myDesc",
"propertyType": {
"_t": "StringPropertyType",
"suggestedValue": "sortOfDefault"
},
"essentiality": {
"_t": "Mandatory"
}
}
```
3. This renders your dataset `ds1` unusable because it lacks the required key-value in its properties
4. Fix the dataset by providing the required key-value by
`PUT http://localhost:8080/menas/api/dataset/ds1/properties` with a payload
```json
{
"requiredKey1": "valueOfRequiredKey1"
}
```
resulting in version 8 of `ds1` that is usable for std/conf
5. Edit the dataset `ds1` by changing its description resulting in version 9.
6. Dataset `ds1` in version 9, now, has no properties set and is unusable for std/conf processing again (spark-submit will fail with this message)

## Expected behavior
It would be expected that the properties would carry over from v 8 to v 9.

## Comments
This is quite serious in cases where we expect that there may be any properties set in the system, until such time, the problem should not be encountered. That being said, it should be probably fixed sooner than later.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.