mlflow / mlflow/mlflow-example
How to upload an artifact only for selected run_id in MLFlow ?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 297
- Forks
- 658
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am aware that we can set default artifact root option to S3 bucket in mlflow server
mlflow server
--backend-store-uri /mnt/persistent-disk
--default-artifact-root s3://my-mlflow-bucket/
--host 0.0.0.0
By doing this, we are setting default storage for artifacts to S3 buckets for all runs.
As a data scientist, I would be training model multiple times and then comparing using MLFlow Tracking UI. I would always suggest to store artifacts at local file system and only push best model to S3 bucket. Basically I am implementing CI/CD pipeline for MLFlow -sagemaker. As soon as best model pushed to S3 bucket, i would trigger CI/CD pipeline and deploy the model to AWS sagemaker endpoint and then using API Gateway i would create the REST API.
Do we have any such MLFlow API which can be used to push the model only for particular run_id ?
From documentation, i couldn't find any such API.
Regards,
Vackysh
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the mlflow server artifact-root configuration and the tracking API described in the issue. Check whether MLflow supports selecting a single run_id for upload while other artifacts remain local, including the S3 and SageMaker workflow. Done means the supported approach or required API behavior is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100