Process ASCII file with fixed length format
- Vorherrschende Sprache
- Scala
- Sterne
- 170
- Forks
- 96
- Ø Merge
- 57 Min.
- Gemergte PRs (30 T.)
- 2
Beschreibung
## Background [Optional]
I have a need to use an ASCII file that has record segments of fixed length defined. There is no LF/CR kind of characters. The new record starts after every 426bytes. I believe, I should be able to ues Cobrix for this. I am looking for a documentation on what "options" should I specify in my spark.read method. I can see cobrix libraries working very well for ebcdic/binary cobol file. The files that I have are simple ASCII files. These files can grow upto 1GB easily. Spark parllel processsing will be very important for me. I do not have copybook for this defined. However, I know that the structure will be like bello2
Header -- size 100 bytes.
Records Segments -- each 426 bytes
Trailer - size 100 bytes (optional)
## Question
What spark.read options from cobrix can I use to process larger ASCII files with fixed or variable length record segments.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.