Lightning-AI / Lightning-AI/pytorch-lightning

Image Logger

Open
#20,612 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature logger
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Description & Motivation

## Motivation

In various experiments, I find myself attempting to visualize "images" of outputs, gradient covariances, representations etc.

PyTorch lightning has a great feature for tracking these metrics with custom callbacks. However, it requires some extra work in order to merge this behaviour with the logging API.

I would like to propose a new logger based around logging and tracking "images" using matplotlib.

### Pitch

I would like to create a new logger focused on logging these types of images. Similar to the CSVLogger which continually saves the desired images across the training run.

In order to not break the current logger API, it can involve a vacuous "log_metrics" method, and then work similar to the TensorboardLogger with custom methods such as "log_image()".

### Alternatives

Alternatives are incorporating this all into a callback function but it becomes quite messy with keeping track of log directories/versions and file names. It makes more sense to me to have a custom logger which can interact with callbacks if necessary.

Another option is logging the objects directly to tensorboard, but the display of these is more difficult to control.

### Additional context

Expanding on the above, the logger could become more of a "PlottingLogger" in the future in order to not just track real-time visualizations of images/matrices/outputs etc. But also realtime plots of metrics which can be continually updated. This is less essential because tensorboard already incorporates this nicely but in environments where sophisticated loggers are unavailable, this can be a good backup.

cc @lantiga @borda

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing CSVLogger and TensorboardLogger interfaces, especially log_metrics, and compare how a custom log_image method could fit without breaking the logger API. Define the logger's image-saving behavior, directory and version handling, and callback interaction before implementation; done means a coherent logger design with documented usage and coverage for the proposed API.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.