AbsaOSS / AbsaOSS/spark-data-standardization

Standardization fails if default value is not defined for a non-nullable array

未關閉
#86 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Scala
星號
7
分支
2
平均合併
1 天 18 小時
30 天內合併 PR
4

描述

## Describe the bug
The error is:
```
Caused by: java.lang.IllegalStateException: No default value defined for data type array
at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValue(CommonTypeDefaults.scala:63) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
at za.co.absa.standardization.types.CommonTypeDefaults.$anonfun$getDataTypeDefaultValueWithNull$1(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
at scala.util.Try$.apply(Try.scala:213) ~[scala-library-2.12.18.jar:?]
at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValueWithNull(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
```

But the natural default value for an array is the empty array.

## To Reproduce
1. Create a schema with a non-nullable array column.
2. Try to standardize data with this schema while not specifying a default value for the array column.
3. See the error.

## Expected behavior
Standardization should use empty array as the default value for array type automatically.

## Business Value
--

## Screenshots
--

## Additional context
--

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。