Lightning-AI / Lightning-AI/pytorch-lightning
`ModelCheckpoint` and `SaveConfigCallback` have different saving path.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
Description & Motivation
I'm using LightningCLI and NeptuneLogger.
When I pass a NeptuneLogger object to Trainer class, I expect the checkpoints and config files to be saved in a path determined by the logger by default. However, the current behavior differs from my expectation.
I slightly investigated how these paths are determined. If a logger is given in a trainer, the ModelCheckpoint.dirpath is determined as shown in the code below.
https://github.com/Lightning-AI/lightning/blob/fe9e5d55bf7991ba36b76d6adae9075b93dfcaa0/src/pytorch_lightning/callbacks/model_checkpoint.py#L605
So I expect the config.yaml will be saved in os.path.join(save_dir, str(name), version). Basically, in NeptuneLogger, the version denotes an automatically assigned ID from Neptune.ai, so collaborating with version is very important.
However, the SaveConfigCallback acts as follows,
https://github.com/Lightning-AI/lightning/blob/6df43685ee2f2dd0c53eefec295e0d6c79796fd2/src/lightning/pytorch/cli.py#L243-L246
which is the same as just save_dir.
Pitch
config.yaml should be saved in os.path.join(save_dir, str(name), version).
Alternatives
None
Additional context
It is very complex to figure out the meaning, behavior, and connections of these paths, such as Trainer.default_root_dir, Trainer.log_dir, ModelCheckpoint.dirpath, 'Logger.log_dir, and Logger.save_dir. It would be nice for users if Lightning provided tutorials about it.
cc @lantiga @borda @mauvilsa
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked sections of src/pytorch_lightning/callbacks/model_checkpoint.py and src/lightning/pytorch/cli.py to compare how ModelCheckpoint.dirpath and SaveConfigCallback choose their paths. Trace the logger values used by NeptuneLogger, then verify that config.yaml follows the same save_dir/name/version location as checkpoints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100