meta-pytorch / meta-pytorch/data
[Nodes] Drop-in Replacement for DataLoader and StatefulDataLoader
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
Provide users utility functions to have drop-in replacement for the well-known torch.utils.data.DataLoader, as well as StatefulDataLoader.
Motivation, pitch
We should be able to re-construct exactly (or pretty darn close) the behaviour of torch.utils.DataLoader and torchdata.StatefulDataLoader by configuring Nodes in a certain way. We should provide these utilities for users as something practical, and as a starting point/reference implementation to get started and make tweaks as they need.
Alternatives
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no repository files or tests. Start by reviewing torch.utils.data.DataLoader and torchdata.StatefulDataLoader, then identify how Nodes could be configured to reproduce their behavior. Done means providing utility functions with behavior that is exactly, or nearly, equivalent to both loaders.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100