AbsaOSS / AbsaOSS/cobrix

EBCDIC Data Columns showing null for PIC 9(5) COMP-3

Đang mở
#308 5 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Scala
Star
170
Fork
96
Merge trung bình
57 phút
Pull request đã merge (30 ngày)
2

Mô tả

**## Describe the bug**
Here are the Cobol copy book and ebcdic data files. I try to read using below statement. I was able to print the schema, however I have following 3 issues.

1. Not able to print CUST_ID , its showing as null. In the sample copyboock CUST_ID is PIC 9(5) COMP-3
2. Not able to parse other columns as per the copy book column fields length.
3. There are total 3 records in data file. but when we display its only showing single record.
However I used data stage I was able to parse and see the data as expected using same copy book and data.

Code:

val df = spark.read.format("cobol").option("schema_retention_policy", "collapse_root").option("is_record_sequence", true).option("copybook", "/myapp/new/sample_mainframe_fb_data_copybook.cob.txt").load("/myapp/new/sample_mainframe_fb_data.dat.txt")
[sample_mainframe_fb_data_copybook.cob.txt](https://github.com/AbsaOSS/cobrix/files/4899450/sample_mainframe_fb_data_copybook.cob.txt)
[sample_mainframe_fb_data.dat.txt](https://github.com/AbsaOSS/cobrix/files/4899460/sample_mainframe_fb_data.dat.txt)

## To Reproduce
Steps to reproduce the behaviour OR commands run:
1. Copy the Cobol and data files to your file system
2. Use the above code snippet in any of your running spark/scala program or spark-shell for interactive execution.

## Expected behaviour
A clear and concise description of what you expected to happen.
1. It should list all 3 records. 2. CUST_ID value able to readable, 3. View and align the other data columns as per the Cobol copybook

## Screenshots
If applicable, add screenshots to help explain your problem.

Results are showing like below

> ### +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> |CUST_ID| CUST_NAME| STREET_ADDRESS| CITY|US_STATE|OTHER_STATE_PROVINCE|COUNTRY_CODE|POSTAL_CODE| NOTES|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> | null|[John, 1 Ma]|in St. Anyt|own NY| | USA1| 234| 5 This|is our first cust...|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+

## Additional context
Add any other context about the problem here.

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.