Lightning-AI / Lightning-AI/pytorch-lightning

Add multiple ModelCheckpoint callbacks support to WandbLogger and adjust model file namings

Open
#18,576 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature logger: wandb
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Description & Motivation

Hi,
I would love to know what you guys think about this. I have already made an implementation for my own purposes, but I would also love to make a PR out of it.

### Pitch

To the best of my knowledge, the current implementation of WandbLogger does not support working with multiple ModelCheckpoint callbacks, which allows logging checkpoints based on multiple metrics. Moreover, the current naming of the logged model checkpoint (using WandbLogger) is using some awkward (in my opinion) naming (something like `model-{self.experiment.id}`). I believe that giving some more naming options can make the experience of working with the logger more user-friendly. For example, we can let the user choose whether to use the experiment's `name` as part of the model's name, or just provide some name prefix or suffix, or even a template, by him/herself.
I find it really convenient.

### Alternatives

_No response_

### Additional context

_No response_

cc @borda @awaelchli @morganmcg1 @borisdayma @scottire @parambharat

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the WandbLogger and ModelCheckpoint implementations to understand how checkpoint callbacks and logged model filenames currently interact. Define the supported behavior for multiple callbacks and user-controlled naming, then verify that the resulting logger behavior is covered by tests; the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.