EBCDIC Data Columns showing null for PIC 9(5) COMP-3
- Linguagem predominante
- Scala
- Estrelas
- 170
- Forks
- 96
- Merge médio
- 57min
- PRs com merge (30d)
- 2
Descrição
**## Describe the bug**
Here are the Cobol copy book and ebcdic data files. I try to read using below statement. I was able to print the schema, however I have following 3 issues.
1. Not able to print CUST_ID , its showing as null. In the sample copyboock CUST_ID is PIC 9(5) COMP-3
2. Not able to parse other columns as per the copy book column fields length.
3. There are total 3 records in data file. but when we display its only showing single record.
However I used data stage I was able to parse and see the data as expected using same copy book and data.
Code:
val df = spark.read.format("cobol").option("schema_retention_policy", "collapse_root").option("is_record_sequence", true).option("copybook", "/myapp/new/sample_mainframe_fb_data_copybook.cob.txt").load("/myapp/new/sample_mainframe_fb_data.dat.txt")
[sample_mainframe_fb_data_copybook.cob.txt](https://github.com/AbsaOSS/cobrix/files/4899450/sample_mainframe_fb_data_copybook.cob.txt)
[sample_mainframe_fb_data.dat.txt](https://github.com/AbsaOSS/cobrix/files/4899460/sample_mainframe_fb_data.dat.txt)
## To Reproduce
Steps to reproduce the behaviour OR commands run:
1. Copy the Cobol and data files to your file system
2. Use the above code snippet in any of your running spark/scala program or spark-shell for interactive execution.
## Expected behaviour
A clear and concise description of what you expected to happen.
1. It should list all 3 records. 2. CUST_ID value able to readable, 3. View and align the other data columns as per the Cobol copybook
## Screenshots
If applicable, add screenshots to help explain your problem.
Results are showing like below
> ### +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> |CUST_ID| CUST_NAME| STREET_ADDRESS| CITY|US_STATE|OTHER_STATE_PROVINCE|COUNTRY_CODE|POSTAL_CODE| NOTES|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
> | null|[John, 1 Ma]|in St. Anyt|own NY| | USA1| 234| 5 This|is our first cust...|
> +-------+------------+--------------------+------------------+--------+--------------------+------------+-----------+--------------------+
## Additional context
Add any other context about the problem here.
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.