Project-MONAI / Project-MONAI/MONAILabel

Simplify adding static parameters and metric to MLFlow

Open
#1,383 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
891
Forks
269
Avg merge
15h 41m
Merged PRs (30d)
1

Description

MLflow feature tracking is great feature that allows user to run multiple experiments and see why.

To do so we must be able to log all metric and parameters used in the experiment. It is not clear/simple to add any static variable to the experiment. For example I have added a lot of parameters which I can control from slicer UI. However, I don't see an easy way to add them to mlflow.

Also I recently added a running val metric val_los how can I add that to the tracked variables

Describe the solution you'd like
It would be great if :
1- is there is a simple method to override and add new static global vars as form the config.
2- there is a simple method to override to add new running metrices
2- we can add all existing parameters as number of gpus used, datacaching used, loaders, time it took to run, etc

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points; start by locating the MLflow integration and the configuration and metric-logging paths. Define the supported way to add static parameters and running metrics, plus which existing run metadata should be recorded, then add coverage for the agreed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.