Lightning-AI / Lightning-AI/pytorch-lightning

Progress bar for non interactive environments

Open
#12,185 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature progress tracking (internal)
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

## 🚀 Feature

I'd like to request a progress bar for non-terminal, non-interactive environments, like for example Amazon CloudWatch. The TQDM and Rich progress bars are not ideal for tracking progress in these more simple logging environments.

Additionally, I'd like to request the ability to automatically adapt to a known-length or an unknown length iterable dataset, by displaying percentage based progress in the former case, and simply iteration count in the latter case. (It appears that the default TQDM progress bar produced errors for me with an unknown length iterable webdataset.)

### Motivation

I'm training on both EC2 and Amazon SageMaker, and neither the TQDM nor the Rich progress bars seem appropriate for monitoring progress in CloudWatch. I asked on slack if a more appropriate alternative or configuration setting was available, and was advised that there's not anything currently, but I should create a GitHub issue for one.

### Pitch

At the most basic level, the progress indicator would print something like this on rank 0:
`[Epoch 0 | Iteration 100] train_loss: 0.01, val_acc: 0.74`

Another useful feature would be optionally passing a logger to use instead of print, so for example if you pass a logger that writes to stdout:
`LoggerName - INFO - [Epoch 0 | Iteration 100] train_loss: 0.01, val_acc: 0.74`

If the epoch has a known total number of batches, additional information on the completion percentage would be nice. For example:
`[Epoch 0 | Iteration 100/450 (22.22% complete)] train_loss: 0.01, val_acc: 0.74`

### Alternatives

I guess whenever anyone needs more simple progress tracking they could write their own custom progress meter based on `ProgressBarBase`.

cc @borda @carmocca @awaelchli

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing ProgressBarBase and the existing TQDM and Rich progress-bar options mentioned in the issue. Define how non-interactive output should handle known- and unknown-length iterables, optional logging, and rank-0 reporting; done would require agreed behavior and tests for those cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, pytorch
Domain
machine-learning, observability, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.