AbsaOSS / AbsaOSS/cobrix

Process ASCII file with fixed length format

未關閉
#651 5 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
question
主要語言
Scala
星號
170
分支
96
平均合併
57 分鐘
30 天內合併 PR
2

描述

## Background [Optional]
I have a need to use an ASCII file that has record segments of fixed length defined. There is no LF/CR kind of characters. The new record starts after every 426bytes. I believe, I should be able to ues Cobrix for this. I am looking for a documentation on what "options" should I specify in my spark.read method. I can see cobrix libraries working very well for ebcdic/binary cobol file. The files that I have are simple ASCII files. These files can grow upto 1GB easily. Spark parllel processsing will be very important for me. I do not have copybook for this defined. However, I know that the structure will be like bello2
Header -- size 100 bytes.
Records Segments -- each 426 bytes
Trailer - size 100 bytes (optional)

## Question
What spark.read options from cobrix can I use to process larger ASCII files with fixed or variable length record segments.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。