Path not filled for root values in Schema upload
- Ngôn ngữ chính
- Scala
- Star
- 33
- Fork
- 16
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
## Describe the bug
`path` in `SchemaField` for the root fields is not filled/populated during upload.
It is not validated in any way. Now we have old schemas with this path and new without. So sometimes `getAbsolutePath` works and sometimes it returns duplicates.
Most probably the root of the cause is in `SparkMenasSchemaConvertor#convertSparkToMenasFields`
This also makes trouble for the export/import feature. Currently, we have reached a stable enough point. The fix should come with a change to the SQL database and a full model audit.
### Expected
```scala
private val schemaFieldRoot = SchemaField(
name = "AnyStruct",
`type` = "struct",
path = "AnyStruct",
elementType = None,
containsNull = None,
nullable = true,
metadata = Map.empty,
children = Seq(schemaFieldChildOne, schemaFieldChildTwo)
)
```
### Current
```scala
private val schemaFieldRoot = SchemaField(
name = "AnyStruct",
`type` = "struct",
path = "",
elementType = None,
containsNull = None,
nullable = true,
metadata = Map.empty,
children = Seq(schemaFieldChildOne, schemaFieldChildTwo)
)
```
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.