New version creation of a dataset do not carry over the properties
- Ngôn ngữ chính
- Scala
- Star
- 33
- Fork
- 16
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
## Describe the bug
Having a testing required properties key on my local Menas and having such a key-value pair in the properties of a dataset I want to run, I have discovered that if a new version of the dataset is created (e.g. by adding description, adding a conformance rule, ...) the properties set to the previous version do not carry over.
This, in turn, may prevent running standardization/conformance if the system has any required properties keys set.
## To Reproduce
Steps to reproduce the behavior OR commands run:
1. Have a dataset name `ds1` in the latest version `7` that works for std/conf
2. Update your system to require a property `requiredKey1`:
`POST http://localhost:8080/menas/api/properties/datasets` with a payload
```json
{
"name": "requiredKey1",
"description": "myDesc",
"propertyType": {
"_t": "StringPropertyType",
"suggestedValue": "sortOfDefault"
},
"essentiality": {
"_t": "Mandatory"
}
}
```
3. This renders your dataset `ds1` unusable because it lacks the required key-value in its properties
4. Fix the dataset by providing the required key-value by
`PUT http://localhost:8080/menas/api/dataset/ds1/properties` with a payload
```json
{
"requiredKey1": "valueOfRequiredKey1"
}
```
resulting in version 8 of `ds1` that is usable for std/conf
5. Edit the dataset `ds1` by changing its description resulting in version 9.
6. Dataset `ds1` in version 9, now, has no properties set and is unusable for std/conf processing again (spark-submit will fail with this message)
## Expected behavior
It would be expected that the properties would carry over from v 8 to v 9.
## Comments
This is quite serious in cases where we expect that there may be any properties set in the system, until such time, the problem should not be encountered. That being said, it should be probably fixed sooner than later.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.