microsoft / microsoft/foundry-local
Workflow to export & compile Azure AI Foundry fine-tuned open models (Ministral) to ONNX for Foundry Local
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 369
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 39
Description
We have fine-tuned an open-weight model (Ministral) inside Azure AI Foundry (Azure ML). Our end goal is to run this fine-tuned model on local/mobile devices using Foundry Local.
**Problem / Pain Point :**
- Fine-tuning jobs in Azure AI Foundry produce standard PyTorch/Safetensors weights or LoRA adapter files.
- Foundry Local strictly requires models compiled into the ONNX runtime format (.onnx files with inference_model.json).
- Existing documentation primarily focuses on compiling base models directly from Hugging Face rather than exporting custom fine-tuned weights directly from Azure ML job artifacts to Foundry Local.
**Questions:**
Has anyone successfully set up or documented an end-to-end pipeline for this scenario? Specifically looking for guidance or recommended best practices on:
- Artifact Extraction: Recommended steps/SDK calls to pull fine-tuned model artifacts (.safetensors / adapter files) out of Azure AI Foundry / Azure ML job storage.
- ONNX Compilation: Standardized configs or scripts (e.g., using Microsoft Olive or onnxruntime-genai) to fuse LoRA adapters, convert, and quantize (INT4/FP16) locally downloaded weights into the ONNX format required by Foundry Local.
- Automated Tooling: Is there an existing or planned feature within Azure AI Foundry / Foundry Local to automate the ONNX compilation phase directly upon completion of a fine-tuning job?
**Environment Details**
- Base Model: Ministral
- Source: Azure AI Foundry / Azure ML Studio
- Target Runtime: Foundry Local (Mobile / Edge devices)
- Target Precision: INT4 ONNX
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository files, tests, or entry points are named. Start by investigating Azure AI Foundry/Azure ML artifact extraction and the Foundry Local ONNX requirements, then determine whether an end-to-end Ministral pipeline for adapter fusion, conversion, and INT4 compilation can be documented or automated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- ai, machine-learning, mobile-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100