`lr_scheduler.step()` must be called before `optimizer.step()`

@SkafteNicki is already working on this.

Since Dec 17, 2025.

### Bug description

Running a DDP BYOL job on Imagenette using [example LightlySSL code](https://docs.lightly.ai/self-supervised-learning/examples/byol.html), and I get a warning that on PyTorch 1.10+, `optimizer.step()` must precede `lr_scheduler.step()`. Based on ln [346](https://github.com/Lightning-AI/pytorch-lightning/blob/19912d090e5d1b0cc099c1742fdddeb1b5344f27/src/lightning/pytorch/loops/training_epoch_loop.py#L346) and [347](https://github.com/Lightning-AI/pytorch-lightning/blob/19912d090e5d1b0cc099c1742fdddeb1b5344f27/src/lightning/pytorch/loops/training_epoch_loop.py#L357), it looks like that _is_ the case, so not sure why the error is cropping up. Or I might be looking in the wrong places...

### What version are you seeing the problem on?

v2.5

### Reproduced in studio

_No response_

### How to reproduce the bug

```python
# This example requires the following dependencies to be installed:
# pip install lightly

# Note: The model and training settings do not follow the reference settings
# from the paper. The settings are chosen such that the example can easily be
# run on a small dataset with a single GPU.

import copy

import pytorch_lightning as pl
import torch
import torchvision
from torch import nn

from lightly.loss import NegativeCosineSimilarity
from lightly.models.modules import BYOLPredictionHead, BYOLProjectionHead
from lightly.models.utils import deactivate_requires_grad, update_momentum
from lightly.transforms.byol_transform import (
BYOLTransform,
BYOLView1Transform,
BYOLView2Transform,
)
from lightly.utils.scheduler import cosine_schedule

class BYOL(pl.LightningModule):
def __init__(self):
super().__init__()
resnet = torchvision.models.resnet18()
self.backbone = nn.Sequential(*list(resnet.children())[:-1])
self.projection_head = BYOLProjectionHead(512, 1024, 256)
self.prediction_head = BYOLPredictionHead(256, 1024, 256)

self.backbone_momentum = copy.deepcopy(self.backbone)
self.projection_head_momentum = copy.deepcopy(self.projection_head)

deactivate_requires_grad(self.backbone_momentum)
deactivate_requires_grad(self.projection_head_momentum)

self.criterion = NegativeCosineSimilarity()

def forward(self, x):
y = self.backbone(x).flatten(start_dim=1)
z = self.projection_head(y)
p = self.prediction_head(z)
return p

def forward_momentum(self, x):
y = self.backbone_momentum(x).flatten(start_dim=1)
z = self.projection_head_momentum(y)
z = z.detach()
return z

def training_step(self, batch, batch_idx):
momentum = cosine_schedule(self.current_epoch, 10, 0.996, 1)
update_momentum(self.backbone, self.backbone_momentum, m=momentum)
update_momentum(self.projection_head, self.projection_head_momentum, m=momentum)
(x0, x1) = batch[0]
p0 = self.forward(x0)
z0 = self.forward_momentum(x0)
p1 = self.forward(x1)
z1 = self.forward_momentum(x1)
loss = 0.5 * (self.criterion(p0, z1) + self.criterion(p1, z0))
return loss

def configure_optimizers(self):
return torch.optim.SGD(self.parameters(), lr=0.06)

model = BYOL()

# We disable resizing and gaussian blur for cifar10.
transform = BYOLTransform(
view_1_transform=BYOLView1Transform(input_size=32, gaussian_blur=0.0),
view_2_transform=BYOLView2Transform(input_size=32, gaussian_blur=0.0),
)
dataset = torchvision.datasets.CIFAR10(
"datasets/cifar10", download=True, transform=transform
)
# or create a dataset from a folder containing images or videos:
# dataset = LightlyDataset("path/to/folder", transform=transform)

dataloader = torch.utils.data.DataLoader(
dataset,
batch_size=256,
shuffle=True,
drop_last=True,
num_workers=8,
)

accelerator = "gpu" if torch.cuda.is_available() else "cpu"

trainer = pl.Trainer(max_epochs=10, devices=1, accelerator=accelerator)
trainer.fit(model=model, train_dataloaders=dataloader)
```

### Error messages and logs

```
/opt/conda/lib/python3.11/site-packages/torch/optim/lr_scheduler.py:192: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
```

### Environment

Current environment

```
#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
```

### More info

_No response_

cc @ethanwharris