Project-MONAI / Project-MONAI/MONAILabel
Simplify adding static parameters and metric to MLFlow
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 891
- Forks
- 269
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 1
Description
MLflow feature tracking is great feature that allows user to run multiple experiments and see why.
To do so we must be able to log all metric and parameters used in the experiment. It is not clear/simple to add any static variable to the experiment. For example I have added a lot of parameters which I can control from slicer UI. However, I don't see an easy way to add them to mlflow.
Also I recently added a running val metric val_los how can I add that to the tracked variables
Describe the solution you'd like
It would be great if :
1- is there is a simple method to override and add new static global vars as form the config.
2- there is a simple method to override to add new running metrices
2- we can add all existing parameters as number of gpus used, datacaching used, loaders, time it took to run, etc
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points; start by locating the MLflow integration and the configuration and metric-logging paths. Define the supported way to add static parameters and running metrics, plus which existing run metadata should be recorded, then add coverage for the agreed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- analytics, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100