Binary file with Endianness is expressed as big-endian and field PIC.

we are migrating a process developed in dataStage to scala, using bookshelves. The Copyboock is defined and we manage to read the file in DF with select. But the fields that in the original process we read as binary (Raw) we fail to interpret correctly.

in the DS process, is define like

this is loaded into a database, converting it to hexadecimal, and we can see it, for example so:


but in DF, we do not know how to treat it and we see it like this:

.withColumn("CMAVCRPE_P55_TEXT_ori", (col("CMAVCRPE_P55_TEXT")))
.withColumn("CMAVCRPE_P55_TEXT_hex", hex(col("CMAVCRPE_P55_TEXT")))

Then, we think that we are not performing the reading correctly and that bits are lost.

Thank you