Lightning-AI / Lightning-AI/pytorch-lightning
trainer.test() with given checkpoint logs last epoch instead of checkpoint epoch
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Bug description
Testing from a given checkpoint leads to logging the epoch number of the last checkpoint instead of the checkpoint specified:
```
trainer = Trainer(..., max_epochs=10)
lightning_module = MyLightningModule(...)
datamodule = MyDatamodule()
trainer.fit(lightning_module , datamodule=datamodule)
trainer.test(lightning_module , datamodule=datamodule, ckpt_path="last") # <-- ok: logs correct epoch and step
ckpt_path="/.../checkpoints/epoch=2-step=396.ckpt"
trainer.test(lightning_module , datamodule=datamodule, ckpt_path=ckpt_path) # <-- incorrect: logs last epoch and step
```
The second test logs epoch 10 instead of epoch 2. Similarly, the step number of the second test is incorrect.
### What version are you seeing the problem on?
v2.2.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue on v2.2.1 using the Trainer, checkpoint, and datamodule example in the report, then trace checkpoint loading and epoch/step logging during trainer.test(). Done means an explicit checkpoint such as epoch=2-step=396.ckpt logs epoch 2 and step 396 rather than the final training values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100