Try to use PyTorch DataLoader again
- Dominant language
- Python
- Stars
- 18
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
We replaced `DataLoader` with a simple `MyDataLoader`, which uses our specific batch sampler as it should be (the PyTorch data loader accesses the batch sampler more than it should). Our batch sampler caters correctly for multiple devices, we do not need clever wrapper code (undocumented).
We'd like to understand why `DataLoader` does not work, and ideally get it back, supporting features like prefetching and memory pinning.
**Describe the solution you'd like**
A data loader which works correctly with our batch sampler in the multi-device case, and is based on the PyTorch implementation.
Contributor guide
Research direction
Start by comparing the current MyDataLoader with the PyTorch DataLoader, focusing on how each accesses the batch sampler in the multi-device case. Done means a PyTorch-based data loader works correctly with the project's batch sampler while supporting prefetching and memory pinning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100