AbsaOSS / AbsaOSS/cobrix

Process ASCII file with fixed length format

Abierto
#651 5 comentarios 0 reacciones 0 asignados Ver en GitHub
question
Lenguaje dominante
Scala
Estrellas
170
Forks
96
Merge medio
57 min
PR fusionados (30 d)
2

Descripción

## Background [Optional]
I have a need to use an ASCII file that has record segments of fixed length defined. There is no LF/CR kind of characters. The new record starts after every 426bytes. I believe, I should be able to ues Cobrix for this. I am looking for a documentation on what "options" should I specify in my spark.read method. I can see cobrix libraries working very well for ebcdic/binary cobol file. The files that I have are simple ASCII files. These files can grow upto 1GB easily. Spark parllel processsing will be very important for me. I do not have copybook for this defined. However, I know that the structure will be like bello2
Header -- size 100 bytes.
Records Segments -- each 426 bytes
Trailer - size 100 bytes (optional)

## Question
What spark.read options from cobrix can I use to process larger ASCII files with fixed or variable length record segments.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.