Lightning-AI / Lightning-AI/pytorch-lightning

Modification of the current_epoch attribute or other interesting @properties without setters

Open
#18,804 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Description & Motivation

When saving a checkpoint during training, it would be valuable to have the ability to set the `current_epoch` attribute of the Trainer class. Currently, attempting to set `current_epoch` results in an AttributeError since it lacks a setter. The motivation for this feature is evident in scenarios where users want to resume training from a specific epoch, especially when working with LR Schedulers. For instance, if a checkpoint is saved at epoch 100, and a change in LR occurs at epoch 150, users may want to restart training from epoch 100 to observe the impact of the new LR. This feature would enhance flexibility in managing training progress and experimenting with different configurations.

### Pitch

I propose adding a setter for the `current_epoch` attribute in the Trainer class. This would allow users to programmatically set the training epoch, facilitating the ability to resume training from a specific point. For example:
```python
trainer = Trainer()
# ...
# After loading a checkpoint
checkpoint = torch.load('path/to/checkpoint.ckpt')
trainer.current_epoch = checkpoint['epoch']
# Now the trainer is set to resume training from the specified epoch
```
configurations without unnecessary training repetition.

### Alternatives

An alternative approach would be to provide a method in the Trainer class explicitly designed for setting the current epoch. However, having a setter for `current_epoch` aligns with the intuitive way Python properties are typically used, offering a cleaner and more idiomatic solution.

### Additional context

This feature becomes particularly relevant in situations where experimentation involves frequent changes in LR or other hyperparameters, allowing users to easily explore different configurations without unnecessary training repetition.

cc @borda

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Trainer class's current_epoch property and the checkpoint loading and resume-training entry points. Review existing tests for checkpoint restoration or trainer state, then determine the expected behavior and validation for assigning an epoch; done means the setter supports the requested resume scenario without breaking scheduler or checkpoint handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.