Unexpected behavior for swapped fields in csv with header
- Lingua principale
- Scala
- Stelle
- 33
- Fork
- 16
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## Describe the bug
The difference between field order in CSV and schema causes wrong treatment of data during standardization. Treatment proceeds accord to fields in the schema with no matter what is in the header of CSV.
A specialty of this bug is that default values are not taken according to field order in the schema but correctly according to the field in CSV.
## To Reproduce
Steps to reproduce the behavior:
1. Have schema and CSV with a header.
2. Fields in schema have default values.
3. The order of fields in the schema is different from the order of fields in CSV.
4. Run standardization process
5. In swapped fields, you can see the default values you set.
6. Description of error in errCol says that standardization try to process different value (according to switched fields)
[Swapped_data.zip](https://github.com/AbsaOSS/enceladus/files/3901116/Swapped_data.zip)
[Aligned_data.zip](https://github.com/AbsaOSS/enceladus/files/3901119/aligned_data.zip)
## Expected behavior
Because there is no dependency on field order if the data are in JSON format I expect that behavior has no dependency on the data format and the behavior is the same.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.