Lightning-AI / Lightning-AI/pytorch-lightning
Support `channels_last` with training
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
## 🚀 Feature
I'd like to try out some `channels_last` training to see if it improves performance (https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html)
I'm not entirely sure what the best way to do this with lightning is but I also think it should probably be one of those features that you set on the trainer and it just magically works.
### Motivation
Using the `channels_last` memory format can improve performance in some cases
### Pitch
Add a trainer flag that does whatever is needed for `channels_last` so
```
trainer = pl.Trainer( ..., memory_format='channels_last')
```
or something like that and then before anything happens with training/testing you need to convert the module
```
if self.memory_format == 'channels_last':
lightning_module = lightning_module.to(memory_format=torch.channels_last)
```
and each batch in the train/test/val loops
```
if self.memory_format == 'channels_last':
batch = batch.to(memory_format=torch.channels_last)
lightning_module(batch)
```
### Alternatives
I have no idea but I assume I could do this without changing lightning although I'm not sure how yet
### Additional context
I am not sure I will be able to PR this one but I'm not opposed to trying
cc @borda @carmocca @justusschock @awaelchli
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the Trainer configuration and the module setup, then trace how batches move through the train, validation, and test loops. Review PyTorch's channels_last memory-format guidance linked in the issue. Done means a Trainer option consistently applies the requested memory format to the module and relevant batches during training, validation, and testing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100