AbsaOSS / AbsaOSS/enceladus

Add support for ignoring trailing line(s) in CSV

未关闭
#1,324 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
feature priority: high request Standardization
主要语言
Scala
星标
33
派生
16
PR 合并指标
30 天内没有已合并 PR

描述

## Background
Sometimes input raw files contain a trailing line that needs to be ignored. We have a use case for the trailing 1 line, but if we need to implement this feature anyway, maybe let's consider do it more generally.

## Feature
Add support for a command-line option to ignore trailing lines of each input CSV file.

## Example [Optional]
```
a,b,c
1,2,3
5,6,7
Total=2
```

## Proposed Solution [Optional]
If there is no such option in `spark-csv`, possibly we need to preprocess incoming CSV files in RDDs.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。