Add validation that output column should be at the same array level
- Lingua principale
- Scala
- Stelle
- 33
- Fork
- 16
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## Describe the bug
For a schema like this:
```
root
|-- id: long (nullable = true)
|-- array1: array (nullable = true)
| |-- element: struct (containsNull = false)
| | |-- key1: long (nullable = true)
| | |-- key2: long (nullable = true)
```
if a join condition for a mapping rule includes `key1` and.or `key2`, then the output column should be inside `array1`. It cannot be at root level. Otherwise the error will occur:
```
Caused by: org.apache.spark.sql.AnalysisException: cannot resolve 'UDF(array1.key1, array1.key2)' due to data type mismatch: argument 2 requires long type, however, 'array1.key' is of array type.;;
```
## Expected behaviour
This case should be validated by Conformance before actually executing a mapping rule.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.