AbsaOSS / AbsaOSS/cobrix

NOT DIVISIBLE by the RECORD SIZE

未關閉
#182 7 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Scala
星號
170
分支
96
平均合併
57 分鐘
30 天內合併 PR
2

描述

Hi @yruslan ,

I am trying to read a file with the following command:
`spark.read.format("cobol").option("copybook", file://BOOK.txt").load("DATASET.dat").count()`

The error output is:

> ERROR FileUtils$: File hdfs://NHA/user/big/DATASET.dat IS NOT divisible by 200.
> java.lang.IllegalArgumentException: There are some files in DATASET.dat that are NOT DIVISIBLE by the RECORD SIZE calculated from the copybook (200 bytes per record). Check the logs for the names of the files.
>

After trying some options, the following worked:
`spark.read.format("cobol").option("copybook", "file://BOOK.txt").option("file_start_offset", "1").load("DATASET.dat ").count()`

But the line count of the file shows only 20.314.408 lines while the expected value is 38.694.112 lines and the column values are completely wrong.

Thanks for your help !

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。