huggingface / huggingface/datasets
Enable splits during streaming the dataset
Open
enhancement
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 5d 7h
- Merged PRs (30d)
- 17
Description
## Describe the Problem
I'd like to stream only a specific percentage or part of the dataset.
I want to do splitting when I'm streaming dataset as well.
## Solution
Enabling splits when `streaming = True` as well.
`e.g. dataset = load_dataset('dataset', split='train[:100]', streaming = True)`
## Alternatives
Below is the alternative of doing it.
`dataset = load_dataset("dataset", split='train', streaming = True).take(100)`
Contributor guide
Assessment
This issue has not been assessed yet.