AbsaOSS / AbsaOSS/cobrix

Process ASCII file with fixed length format

Aberta
#651 5 comentários 0 reações 0 responsáveis Ver no GitHub
question
Linguagem predominante
Scala
Estrelas
170
Forks
96
Merge médio
57min
PRs com merge (30d)
2

Descrição

## Background [Optional]
I have a need to use an ASCII file that has record segments of fixed length defined. There is no LF/CR kind of characters. The new record starts after every 426bytes. I believe, I should be able to ues Cobrix for this. I am looking for a documentation on what "options" should I specify in my spark.read method. I can see cobrix libraries working very well for ebcdic/binary cobol file. The files that I have are simple ASCII files. These files can grow upto 1GB easily. Spark parllel processsing will be very important for me. I do not have copybook for this defined. However, I know that the structure will be like bello2
Header -- size 100 bytes.
Records Segments -- each 426 bytes
Trailer - size 100 bytes (optional)

## Question
What spark.read options from cobrix can I use to process larger ASCII files with fixed or variable length record segments.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.