AbsaOSS / AbsaOSS/cobrix

Can I get the raw record bytes from ebcdic file w/out parsing

Offen
#656 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
question
Vorherrschende Sprache
Scala
Sterne
170
Forks
96
Ø Merge
57 Min.
Gemergte PRs (30 T.)
2

Beschreibung

I am trying to parse an ebcdic file for which I do not have a copybook. I know that whether it has RDW and/or BDW. It is one of the old legacy format file. We have written our own program that knows how to parse an individual record.

Is there a way that I can use Cobrix library only to parse and individual record in ebcdic bytes ? Once I get those bytes in rdd, I can write my 'map' function to parse the individual segments. I have used Cobrix library get an individual record. I have used following setup,

I have defined my copybook in a simple structute like below,

```
String copybook =
" 01 RECORD.\n" +
" 05 SEGMENT PIC X(1064).\n" ;

Dataset df1 = spark.read()
.format("za.co.absa.cobrix.spark.cobol.source")
.option("copybook_contents", copybook)
.option("encoding", "ebcdic")
.option("record_format", "V") // Variable length records
.option("is_rdw_big_endian", "true")
.option("rdw_adjustment", -4)
.option("schema_retention_policy", "collapse_root")
.load("data/Samples/sample_packed_variable_ebcdic_bigendian_rdw.dat");

```

I am able to parse the records based on the "RDW" value correctly. I get ROW object with only one element in it (as specified in my copybook with name SEGMENT). This SEGMENT is coming as string. I convert this string in "ibm500" character code set (convert to ebcdic) and parse it with the parsing program that we have written. Our program can parse the record based on the byte position. However, we are not able to parse the 'packed' decimal properly. It seems that the conversion from string to/from ebcdic bytes is losing the position and format. Is there a way for us to get origina raw bytes format as it appears in the file as part of the dataset that we get out. In shore, can 'SEGMENT' field in my example, represent the actual raw bytes for the entire record in the file ?

## Question
Is above method an acceptable way that I can use this library ? I like the Corbix's way of parallel processing of reading large ebcdic file and all I want help from this library is to parse rdw/bdw value and return the entire record in raw bytes that I can use for my own parsing logic. As I do not have proper copybood for the bytes segment.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.