dask read_csv ignore broken files
Open
dataframe
feature
io
needs attention
- Dominant language
- Python
- Stars
- 13.9k
- Forks
- 2k
- PR merge metrics
- No merged PRs in 30d
Description
I try to read the CSV files with,
```python
import dask.dataframe as dd
data = dd.read_csv("*.csv.gz",sep='\t',dtype='unicode', low_memory='true',compression='gzip',blocksize=None)
```
I have some CSV files which are broken files. I want to ignore that. It will be great if we have some function like below,
```python
import dask.dataframe as dd
data = dd.read_csv("*.csv.gz",sep='\t',dtype='unicode', low_memory='true',compression='gzip',blocksize=None,ignore_broken_files = True)
```
Which will skip the files that can't be read.
Contributor guide
Assessment
This issue has not been assessed yet.