Lightning-AI / Lightning-AI/pytorch-lightning

Support for returning LRSchedulerConfig on LightningModule.configure_optimizers

Open
#18,870 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature help wanted lightningmodule lr scheduler
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

Description & Motivation

This is a re-proposal for #13569. At that time, this feature is considered to be discouraged since LRSchedulerConfig is supposed to be an "internal structure" (according to this comment). However, as #18463 adds type hints for the returns of configure_optimizers (also can be found in the current documentation), one of them is Tuple[Sequence[Optimizer], Sequence[Union[LRScheduler, ReduceLROnPlateau, LRSchedulerConfig]]], indicating that returning LRSchedulerConfig should be supported now.

According to my personal experience, I find returning LRSchedulerConfig handy, and I hope this feature could be supported, instead of revising the type hints. Thanks!

Pitch

The suggestion proposed in #13569 still should work, i.e., adding the following two lines to _configure_schedulers_automatic_opt:

...
elif isinstance(scheduler, LRSchedulerConfig):
    config = scheduler
...
Alternatives

No response

Additional context

No response

cc @borda @carmocca @justusschock @awaelchli

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at LightningModule.configure_optimizers and trace how _configure_schedulers_automatic_opt handles scheduler values, especially LRSchedulerConfig. Compare the existing scheduler branches with the proposed handling, then verify that returning LRSchedulerConfig is accepted without changing other supported return forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.