Lightning-AI / Lightning-AI/pytorch-lightning
Redesign dataloader re-instantiation context managers for Lite
@awaelchli is already working on this.
Since Dec 12, 2022.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
## Proposed refactor
Redesign the context managers that enable re-instantiations of DataLoader after #14992 deprecates the run method design.
### Motivation
We currently have two context managers over the run method:
https://github.com/Lightning-AI/lightning/blob/776432fd7e34cc02e9dbfb16ad8ec8491c176dbf/src/lightning_lite/lite.py#L394-L397
- _replace_dunder_methods(DataLoader, "dataset")
- _replace_dunder_methods(BatchSampler)
These get applied for the user if they implement the run method. However, this won't be the case anymore after #14992. We need to find an alternative approach.
### Pitch
We probably need to call these methods at the start of the program, for example in `LightningLite.__init__`. But it is unclear how to undo the patching which is usually done in `__exit__`. Happy to hear suggestions.
cc @borda @justusschock @awaelchli @tchaton @carmocca @otaj
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.