microsoft / microsoft/foundry-local

Workflow to export & compile Azure AI Foundry fine-tuned open models (Ministral) to ONNX for Foundry Local

Open
#897 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
2.6k
Forks
369
Avg merge
2d 17h
Merged PRs (30d)
39

Description

We have fine-tuned an open-weight model (Ministral) inside Azure AI Foundry (Azure ML). Our end goal is to run this fine-tuned model on local/mobile devices using Foundry Local.

**Problem / Pain Point :**
- Fine-tuning jobs in Azure AI Foundry produce standard PyTorch/Safetensors weights or LoRA adapter files.
- Foundry Local strictly requires models compiled into the ONNX runtime format (.onnx files with inference_model.json).
- Existing documentation primarily focuses on compiling base models directly from Hugging Face rather than exporting custom fine-tuned weights directly from Azure ML job artifacts to Foundry Local.

**Questions:**
Has anyone successfully set up or documented an end-to-end pipeline for this scenario? Specifically looking for guidance or recommended best practices on:
- Artifact Extraction: Recommended steps/SDK calls to pull fine-tuned model artifacts (.safetensors / adapter files) out of Azure AI Foundry / Azure ML job storage.
- ONNX Compilation: Standardized configs or scripts (e.g., using Microsoft Olive or onnxruntime-genai) to fuse LoRA adapters, convert, and quantize (INT4/FP16) locally downloaded weights into the ONNX format required by Foundry Local.
- Automated Tooling: Is there an existing or planned feature within Azure AI Foundry / Foundry Local to automate the ONNX compilation phase directly upon completion of a fine-tuning job?

**Environment Details**
- Base Model: Ministral
- Source: Azure AI Foundry / Azure ML Studio
- Target Runtime: Foundry Local (Mobile / Edge devices)
- Target Precision: INT4 ONNX

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository files, tests, or entry points are named. Start by investigating Azure AI Foundry/Azure ML artifact extraction and the Foundry Local ONNX requirements, then determine whether an end-to-end Ministral pipeline for adapter fusion, conversion, and INT4 compilation can be documented or automated.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ai, machine-learning, mobile-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.