Can I get the raw record bytes from ebcdic file w/out parsing
- Lenguaje dominante
- Scala
- Estrellas
- 170
- Forks
- 96
- Merge medio
- 57 min
- PR fusionados (30 d)
- 2
Descripción
I am trying to parse an ebcdic file for which I do not have a copybook. I know that whether it has RDW and/or BDW. It is one of the old legacy format file. We have written our own program that knows how to parse an individual record.
Is there a way that I can use Cobrix library only to parse and individual record in ebcdic bytes ? Once I get those bytes in rdd, I can write my 'map' function to parse the individual segments. I have used Cobrix library get an individual record. I have used following setup,
I have defined my copybook in a simple structute like below,
```
String copybook =
" 01 RECORD.\n" +
" 05 SEGMENT PIC X(1064).\n" ;
Dataset df1 = spark.read()
.format("za.co.absa.cobrix.spark.cobol.source")
.option("copybook_contents", copybook)
.option("encoding", "ebcdic")
.option("record_format", "V") // Variable length records
.option("is_rdw_big_endian", "true")
.option("rdw_adjustment", -4)
.option("schema_retention_policy", "collapse_root")
.load("data/Samples/sample_packed_variable_ebcdic_bigendian_rdw.dat");
```
I am able to parse the records based on the "RDW" value correctly. I get ROW object with only one element in it (as specified in my copybook with name SEGMENT). This SEGMENT is coming as string. I convert this string in "ibm500" character code set (convert to ebcdic) and parse it with the parsing program that we have written. Our program can parse the record based on the byte position. However, we are not able to parse the 'packed' decimal properly. It seems that the conversion from string to/from ebcdic bytes is losing the position and format. Is there a way for us to get origina raw bytes format as it appears in the file as part of the dataset that we get out. In shore, can 'SEGMENT' field in my example, represent the actual raw bytes for the entire record in the file ?
## Question
Is above method an acceptable way that I can use this library ? I like the Corbix's way of parallel processing of reading large ebcdic file and all I want help from this library is to parse rdw/bdw value and return the entire record in raw bytes that I can use for my own parsing logic. As I do not have proper copybood for the bytes segment.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.