AbsaOSS / AbsaOSS/cobrix

EBCDIC Data Columns showing null for PIC 9(5) COMP-3

未关闭
#308 5 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
bug
主要语言
Scala
星标
170
派生
96
平均合并
57 分钟
30 天内合并 PR
2

描述

**## Describe the bug**
Here are the Cobol copy book and ebcdic data files. I try to read using below statement. I was able to print the schema, however I have following 3 issues.

1. Not able to print CUST_ID , its showing as null. In the sample copyboock CUST_ID is PIC 9(5) COMP-3
2. Not able to parse other columns as per the copy book column fields length.
3. There are total 3 records in data file. but when we display its only showing single record.
However I used data stage I was able to parse and see the data as expected using same copy book and data.

Code:

val df = spark.read.format("cobol").option("schema_retention_policy", "collapse_root").option("is_record_sequence", true).option("copybook", "/myapp/new/sample_mainframe_fb_data_copybook.cob.txt").load("/myapp/new/sample_mainframe_fb_data.dat.txt")
[sample_mainframe_fb_data_copybook.cob.txt](https://github.com/AbsaOSS/cobrix/files/4899450/sample_mainframe_fb_data_copybook.cob.txt)
[sample_mainframe_fb_data.dat.txt](https://github.com/AbsaOSS/cobrix/files/4899460/sample_mainframe_fb_data.dat.txt)

## To Reproduce
Steps to reproduce the behaviour OR commands run:
1. Copy the Cobol and data files to your file system
2. Use the above code snippet in any of your running spark/scala program or spark-shell for interactive execution.

## Expected behaviour
A clear and concise description of what you expected to happen.
1. It should list all 3 records. 2. CUST_ID value able to readable, 3. View and align the other data columns as per the Cobol copybook

## Screenshots
If applicable, add screenshots to help explain your problem.

Results are showing like below

> ### +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> |CUST_ID| CUST_NAME| STREET_ADDRESS| CITY|US_STATE|OTHER_STATE_PROVINCE|COUNTRY_CODE|POSTAL_CODE| NOTES|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> | null|[John, 1 Ma]|in St. Anyt|own NY| | USA1| 234| 5 This|is our first cust...|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+

## Additional context
Add any other context about the problem here.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。