Lightning-AI / Lightning-AI/pytorch-lightning
Add multiple ModelCheckpoint callbacks support to WandbLogger and adjust model file namings
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Description & Motivation
Hi,
I would love to know what you guys think about this. I have already made an implementation for my own purposes, but I would also love to make a PR out of it.
### Pitch
To the best of my knowledge, the current implementation of WandbLogger does not support working with multiple ModelCheckpoint callbacks, which allows logging checkpoints based on multiple metrics. Moreover, the current naming of the logged model checkpoint (using WandbLogger) is using some awkward (in my opinion) naming (something like `model-{self.experiment.id}`). I believe that giving some more naming options can make the experience of working with the logger more user-friendly. For example, we can let the user choose whether to use the experiment's `name` as part of the model's name, or just provide some name prefix or suffix, or even a template, by him/herself.
I find it really convenient.
### Alternatives
_No response_
### Additional context
_No response_
cc @borda @awaelchli @morganmcg1 @borisdayma @scottire @parambharat
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the WandbLogger and ModelCheckpoint implementations to understand how checkpoint callbacks and logged model filenames currently interact. Define the supported behavior for multiple callbacks and user-controlled naming, then verify that the resulting logger behavior is covered by tests; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100