Lightning-AI / Lightning-AI/pytorch-lightning
Please allow automatic optimization for multiple optimizers again.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Description & Motivation
I'm suggesting allowing the old behavior to work again. While still giving users the option to use the new behavior if they set self.automatic_optimization=False. The old API was well designed and can allow for extremely simple implementations especially in situations where the training step is the same for each optimizer being used (i.e. no optimizer_idx if-statement).
### Pitch
The original purpose of Pytorch-Lightning was to simplify & eliminate the boiler plate in the pytorch training loop. But the new behavior is **much more complicated than even using base pytorch**. Since it requires extra bloat like `self.automatic_optimization=False`, `self.toggle_optimizer()`, `self.untoggle_optimizer()`, `self.optimizers()`, then using custom rewrites of well-known base pytorch APIs like `self.manual_backwards()`, in addition to reintroducing the boiler-plate that Pytorch-Lightning was made to remove.
As a matter of fact **in the simplest case it adds 12 additional lines of bloat...**
### Alternatives
_No response_
### Additional context
Could you at least consider collecting user feedback before you remove useful features like this in the future?
cc @borda
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the automatic_optimization and multiple-optimizer behavior described in the issue, including the manual APIs listed: toggle_optimizer(), untoggle_optimizer(), optimizers(), and manual_backwards(). Trace the training loop entry points and existing feedback or tests for optimizer handling. Done would require a decided API design that restores the requested automatic behavior while preserving the opt-out path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100