microsoft / microsoft/onnxruntime

[Feature Request] LoRa adapter support for onnxruntime-web

Open
#26,726 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request platform:mobile platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

# Describe the feature request

Add support for LoRa adapters in onnxruntime-web for inference. So be able to load and use .onnx_adapter files.

See: https://onnxruntime.ai/docs/genai/tutorials/finetune.html

Is this planned?

# Describe scenario use case

## 1. Dynamic Model Switching in the Browser
Allow real-time switching between different fine-tuned behaviors: sentiment analysis, instruction tuning, role personas, or domain specializations by applying different LoRa adapters to a single base ONNX model. This avoids reloading or duplicating the entire model graph.

## 2. Reduced Memory Footprint for Multi-Model Applications
Loading multiple fine-tuned ONNX models currently requires storing several large model files in memory.
With LoRa adapter support, only **one base model** needs to stay loaded, while small LoRa adapters are applied at runtime. This significantly reduces RAM usage in:

- Browser environments
- Mobile or low-memory devices
- Multi-model applications
- Tooling that frequently swaps between fine-tuned variants

This solves the problem of needing multiple full ONNX models to run different fine-tune behaviors.

## 3. Faster Loading and Improved User Experience
LoRa adapters are small, enabling near-instant switching between fine-tuned behaviors without downloading or initializing an entire model again.
This improves UX in:

- Web-based LLM applications
- Interactive tools
- Apps where tasks change frequently (e.g., summarize → classify → generate)

This results in faster startup, reduced bandwidth use, and smoother runtime transitions.

## 4. Lower Bandwidth for Edge Devices
Users on limited networks or mobile connections benefit greatly from downloading small LoRa adapters instead of full multi-GB ONNX models.
Supporting LoRa adapters in onnxruntime-web:

- Cuts bandwidth consumption
- Reduces CDN/storage costs
- Enables on-device personalization without heavy downloads
- Makes large LLMs feasible in constrained environments

This is especially valuable for browser-based AI on edge hardware or mobile web apps.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ONNX Runtime GenAI fine-tuning tutorial linked in the issue and the onnxruntime-web implementation. Determine the browser-facing loading and inference entry points needed for .onnx_adapter files; done means a base ONNX model can load and use LoRa adapters at runtime.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning
Domain
machine-learning, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.