AbsaOSS / AbsaOSS/enceladus

Unexpected behavior for swapped fields in csv with header

Đang mở
#1,072 0 bình luận 1 reaction 0 người được giao Xem trên GitHub
bug priority: undecided Standardization
Ngôn ngữ chính
Scala
Star
33
Fork
16
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

## Describe the bug
The difference between field order in CSV and schema causes wrong treatment of data during standardization. Treatment proceeds accord to fields in the schema with no matter what is in the header of CSV.
A specialty of this bug is that default values are not taken according to field order in the schema but correctly according to the field in CSV.

## To Reproduce
Steps to reproduce the behavior:
1. Have schema and CSV with a header.
2. Fields in schema have default values.
3. The order of fields in the schema is different from the order of fields in CSV.
4. Run standardization process
5. In swapped fields, you can see the default values you set.
6. Description of error in errCol says that standardization try to process different value (according to switched fields)

[Swapped_data.zip](https://github.com/AbsaOSS/enceladus/files/3901116/Swapped_data.zip)
[Aligned_data.zip](https://github.com/AbsaOSS/enceladus/files/3901119/aligned_data.zip)

## Expected behavior
Because there is no dependency on field order if the data are in JSON format I expect that behavior has no dependency on the data format and the behavior is the same.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.