AbsaOSS / AbsaOSS/enceladus

Add validation that output column should be at the same array level

Abierto
#1,296 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug Conformance priority: medium
Lenguaje dominante
Scala
Estrellas
33
Forks
16
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Describe the bug
For a schema like this:
```
root
|-- id: long (nullable = true)
|-- array1: array (nullable = true)
| |-- element: struct (containsNull = false)
| | |-- key1: long (nullable = true)
| | |-- key2: long (nullable = true)
```
if a join condition for a mapping rule includes `key1` and.or `key2`, then the output column should be inside `array1`. It cannot be at root level. Otherwise the error will occur:

```
Caused by: org.apache.spark.sql.AnalysisException: cannot resolve 'UDF(array1.key1, array1.key2)' due to data type mismatch: argument 2 requires long type, however, 'array1.key' is of array type.;;
```

## Expected behaviour
This case should be validated by Conformance before actually executing a mapping rule.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.