tensorflow / tensorflow/datasets
Streaming dataset construction or appending to an existing dataset
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Is your feature request related to a problem? Please describe.
I am always frustrated when I need to re-run tfds build when new data samples becomes available. Doing so is time-consuming.
Describe the solution you'd like
The ability to append data to an existing tfds dataset
Describe alternatives you've considered
I am not sure if there are any alternatives
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the TFDS build workflow and how an existing dataset is represented. Define how new samples would be appended without rerunning tfds build, including the expected behavior for dataset construction and completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100