meta-pytorch / meta-pytorch/data
Add option to disable random seed sharding and syntonization for DistributedReadingService
Open
@ejguan is already working on this.
Since Feb 24, 2023.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
This is a customer request for inference, no synchronization is needed because no all_reduce is needed. And, normally inference doesn't require shuffling and requires high throughput. Sharing random seed becomes redundant in this scenario.
We might want to add two more arguments to enable/disable synchronization.
Motivation, pitch
Provide better performance for inference using DataPipe + DataLoader2
Alternatives
Add a special ReadingService for distributed inference.
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.