Lightning-AI / Lightning-AI/pytorch-lightning
[CLI] predict(ckpt_path="best") raises ValueError even though ModelCheckpoint is configured under trainer.callbacks
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Bug description
When running trainer.predict via LightningCLI with --ckpt_path best, Lightning raises:
```bash
ValueError: `.predict(ckpt_path="best")` is set but `ModelCheckpoint` is not configured to save the best model.
```
However, ModelCheckpoint is configured under trainer.callbacks in the YAML, and training is executed via LightningCLI. This looks like either:
1. LightningCLI isn’t detecting the ModelCheckpoint from the CLI YAML when resolving best during predict, or
2. The checkpoint_connector’s detection logic for best doesn’t handle the CLI/YAML path or log directory resolution, or
3. A mismatch between monitor metric registration and best-checkpoint discovery is not surfaced clearly (i.e., the callback exists, but the condition for “best” was silently unmet).
### What version are you seeing the problem on?
v2.5
### Reproduced in studio
_No response_
### How to reproduce the bug
```python
trainer:
callbacks:
- class_path: lightning.pytorch.callbacks.ModelCheckpoint
init_args:
monitor: val_loss
mode: min
save_top_k: 1
filename: "{val_loss:.4f}"
save_last: false
uv run src/cli.py predict -c config.yaml --ckpt_path=best
```
### Error messages and logs
```
# Error messages and logs here please
```
### Environment
Current environment
```
#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
```
### More info
_No response_
cc @lantiga @mauvilsa
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/cli.py and the LightningCLI predict entry point, then trace how --ckpt_path=best reaches the checkpoint_connector. Reproduce with the provided config.yaml and compare trainer.callbacks configuration with best-checkpoint discovery. Done means the supplied ModelCheckpoint configuration is correctly handled, or the failure explains the unmet condition clearly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100