In fread skip_blank_lines default behavior is too strict
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
Stemming from the discussion in #837, we want fread to be more lenient towards empty lines in the input. Currently, `skip_blank_lines` is `False` by default (raises an exception when a blank line is encountered). When set to `True`, the option will cause all empty lines to be skipped.
Ideally, the default behavior should be somewhere in between these two: if an empty line is found then skip it, provided the next line has the "correct" number of columns. Empty lines should not be skipped by default in single-column mode (instead they are treated as NAs or blanks).
In addition, current error message when encountering blank lines is bad: `RuntimeError: Expecting 4 cols but row 0 contains only 0 cols (sep=','). Consider fill=true. ""`. We want to be more explicit, for example: `FreadError: Empty line found on line N in the input, and text exists afterwards. Use skip_blank_lines=True or fill=True to ignore this error.`
Contributor guide
Assessment
This issue has not been assessed yet.