Lightning-AI / Lightning-AI/pytorch-lightning

MLFlowLogger: log system metrics

Open
#20,563 1 comment 14 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature logger: mlflow
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Description & Motivation

I am using the `MLFlowLogger` to keep track of my experiments. To my knowledge, there is no possibility to enable the logging of system metrics: https://mlflow.org/docs/latest/system-metrics/index.html
To me, it seems that both `mlflow.enable_system_metrics_logging()` and the environment variable `MLFLOW_ENABLE_SYSTEM_METRICS_LOGGING` are ignored by `MLFlowLogger`.

Would it be possible to add the option? I think the most sensible would be to add an argument to `MLFlowLogger` mirroring the behavior of `mlflow.start_run(log_system_metrics=True)`.

-----

As I am already asking, would it also be possible to rename/customize the checkpoints of `log_model`? The names contain the epoch as number without leading zeros, such that they are incorrectly sorted in the MLflow interface. I would prefer to have some leading zeros such that lexicographical ordering corresponds to the ordering of the epochs.

### Pitch

_No response_

### Alternatives

_No response_

### Additional context

_No response_

cc @lantiga @borda

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the MLFlowLogger entry point and compare its behavior with mlflow.start_run(log_system_metrics=True), as well as the MLflow system-metrics documentation linked in the issue. Determine how system-metrics logging and checkpoint naming should be exposed, then verify that both requested behaviors work through the logger.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.