meta-pytorch / meta-pytorch/data
[TODO] Separate into function, because we also need to apply distributed seed and call it inside process
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
This issue is generated from the TODO line
https://github.com/pytorch/data/blob/2f29adba451e1b87f1c0c654557d9dd98673fdd8/./torchdata/dataloader2/reading_service.py#L151
cc @VitalyFedyunin
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in torchdata/dataloader2/reading_service.py around line 151, where the TODO identifies the code to separate. Trace how the distributed seed is applied and how process invokes that logic. Done means the logic is in a separate function and is called from process.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, distributed-systems
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100