AbsaOSS / AbsaOSS/cobrix

EBCDIC Data Columns showing null for PIC 9(5) COMP-3

未關閉
#308 5 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Scala
星號
170
分支
96
平均合併
57 分鐘
30 天內合併 PR
2

描述

**## Describe the bug**
Here are the Cobol copy book and ebcdic data files. I try to read using below statement. I was able to print the schema, however I have following 3 issues.

1. Not able to print CUST_ID , its showing as null. In the sample copyboock CUST_ID is PIC 9(5) COMP-3
2. Not able to parse other columns as per the copy book column fields length.
3. There are total 3 records in data file. but when we display its only showing single record.
However I used data stage I was able to parse and see the data as expected using same copy book and data.

Code:

val df = spark.read.format("cobol").option("schema_retention_policy", "collapse_root").option("is_record_sequence", true).option("copybook", "/myapp/new/sample_mainframe_fb_data_copybook.cob.txt").load("/myapp/new/sample_mainframe_fb_data.dat.txt")
[sample_mainframe_fb_data_copybook.cob.txt](https://github.com/AbsaOSS/cobrix/files/4899450/sample_mainframe_fb_data_copybook.cob.txt)
[sample_mainframe_fb_data.dat.txt](https://github.com/AbsaOSS/cobrix/files/4899460/sample_mainframe_fb_data.dat.txt)

## To Reproduce
Steps to reproduce the behaviour OR commands run:
1. Copy the Cobol and data files to your file system
2. Use the above code snippet in any of your running spark/scala program or spark-shell for interactive execution.

## Expected behaviour
A clear and concise description of what you expected to happen.
1. It should list all 3 records. 2. CUST_ID value able to readable, 3. View and align the other data columns as per the Cobol copybook

## Screenshots
If applicable, add screenshots to help explain your problem.

Results are showing like below

> ### +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> |CUST_ID| CUST_NAME| STREET_ADDRESS| CITY|US_STATE|OTHER_STATE_PROVINCE|COUNTRY_CODE|POSTAL_CODE| NOTES|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> | null|[John, 1 Ma]|in St. Anyt|own NY| | USA1| 234| 5 This|is our first cust...|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+

## Additional context
Add any other context about the problem here.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。