AbsaOSS / AbsaOSS/spark-data-standardization

Standardization fails if default value is not defined for a non-nullable array

Abierto
#86 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Scala
Estrellas
7
Forks
2
Merge medio
1 d 18 h
PR fusionados (30 d)
4

Descripción

## Describe the bug
The error is:
```
Caused by: java.lang.IllegalStateException: No default value defined for data type array
at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValue(CommonTypeDefaults.scala:63) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
at za.co.absa.standardization.types.CommonTypeDefaults.$anonfun$getDataTypeDefaultValueWithNull$1(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
at scala.util.Try$.apply(Try.scala:213) ~[scala-library-2.12.18.jar:?]
at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValueWithNull(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
```

But the natural default value for an array is the empty array.

## To Reproduce
1. Create a schema with a non-nullable array column.
2. Try to standardize data with this schema while not specifying a default value for the array column.
3. See the error.

## Expected behavior
Standardization should use empty array as the default value for array type automatically.

## Business Value
--

## Screenshots
--

## Additional context
--

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.