variable length copybook
- Linguagem predominante
- Scala
- Estrelas
- 170
- Forks
- 96
- Merge médio
- 57min
- PRs com merge (30d)
- 2
Descrição
When I tried to parse variable length file with rdw headers. I tried like below
cobolDataframe = spark
.read
.format("cobol")
.option("copybook", v_copybook)
.option("schema_retention_policy", "collapse_root") //removes the root record headerc
.option("drop_group_fillers", "false")
.option("generate_record_id", false) // this adds the file id and record id
.option("is_record_sequence", "true") // reader to use 4 byte record headers to extract records from a mainframe file
.option("is_rdw_big_endian", "true")
.option("is_rdw_part_of_record_length", true)
.option("rdw_adjustment", -4)
.load(v_data)
After parsing the data is not correct. I tried by having different rdw options but in all case I did not get correct parsed data.
Same file if I use with recordlength option
.option("record_length_field",v_recordLengthField)
its parsing correctly. But all of our mainframe files does not have record length field. How can I resolve this. so variable length files should work.
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.