meta-pytorch / meta-pytorch/data
Set better defaults for `MultiProcessingReadingService`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 179
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 2
Description
🚀 The feature
class MultiProcessingReadingService(ReadingServiceInterface):
num_workers: int = get_number_of_cpu_cores()
pin_memory: bool = True
timeout: float
worker_init_fn: Optional[Callable[[int], None]] # Remove this?
prefetch_factor: int = profile_optimal_prefetch_factor(model : nn.Module)
persistent_workers: bool = True
I can add these, opening this issue to discuss whether it's a good idea to change defaults.
+: Users get better out of the box performance with torchdata
-: backward compatibility issues when moving from dataloaderv1 to dataloaderv2
Motivation, pitch
There are many issues on discuss, stack overflow, and blogs describing how people should configure data loaders for optimized performance. Since a lot of the tricks haven't changed like pin_memory = true or num_workers = num_cpu_cores or persistent_workers=true and since we're in the process of developing dataloaderv2 now may be a good time to revisit these default values
- https://www.jpatrickpark.com/post/prefetcher/#:~:text=The%20prefetch_factor%20parameter%20only%20controls,samples%20prefetched%20across%20all%20workers.)
- https://stackoverflow.com/questions/53998282/how-does-the-number-of-workers-parameter-in-pytorch-dataloader-actually-work
- https://discuss.pytorch.org/t/when-to-set-pin-memory-to-true/19723
Alternatives
- Instead of setting reasonable defaults, we can instead extend the
linter.pyto suggest some of these tips if we notice some sources of slowdowns - Do nothing, suggest people read documentation when configuring performance
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating MultiProcessingReadingService and comparing its dataloaderv1 and dataloaderv2 behavior. Review the proposed worker, memory, timeout, initialization, prefetch, and persistence defaults alongside linter.py and the linked performance discussions; done requires an agreed compatibility-safe direction, with tests or documentation covering the decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100