huggingface / huggingface/datasets
Add option to ignore keys/columns when loading a dataset from jsonl(or any other data format)
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 5d 7h
- Merged PRs (30d)
- 17
Description
### Feature request
Hi, I would like the option to ignore keys/columns when loading a dataset from files (e.g. jsonl).
### Motivation
I am working on a dataset which is built on jsonl. It seems the dataset is unclean and a column has different types in each row. I can't clean this or remove the column (It is not my data and it is too big for me to clean and save on my own hardware).
I would like the option to just ignore this column when using `load_dataset`, since i don't need it.
I tried to look if this is already possible but couldn't find a solution. if there is I would love some help. If it is not currently possible, I would love this feature
### Your contribution
I don't think I can help this time, unfortunately.
Contributor guide
Assessment
This issue has not been assessed yet.