mlflow / mlflow/mlflow-example

How to upload an artifact only for selected run_id in MLFlow ?

Open
#6 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
297
Forks
658
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am aware that we can set default artifact root option to S3 bucket in mlflow server

mlflow server
--backend-store-uri /mnt/persistent-disk
--default-artifact-root s3://my-mlflow-bucket/
--host 0.0.0.0

By doing this, we are setting default storage for artifacts to S3 buckets for all runs.
As a data scientist, I would be training model multiple times and then comparing using MLFlow Tracking UI. I would always suggest to store artifacts at local file system and only push best model to S3 bucket. Basically I am implementing CI/CD pipeline for MLFlow -sagemaker. As soon as best model pushed to S3 bucket, i would trigger CI/CD pipeline and deploy the model to AWS sagemaker endpoint and then using API Gateway i would create the REST API.

Do we have any such MLFlow API which can be used to push the model only for particular run_id ?
From documentation, i couldn't find any such API.

Regards,
Vackysh

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the mlflow server artifact-root configuration and the tracking API described in the issue. Check whether MLflow supports selecting a single run_id for upload while other artifacts remain local, including the S3 and SageMaker workflow. Done means the supported approach or required API behavior is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.