microsoft / microsoft/onnxruntime
[Feature Request] LoRa adapter support for onnxruntime-web
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
# Describe the feature request
Add support for LoRa adapters in onnxruntime-web for inference. So be able to load and use .onnx_adapter files.
See: https://onnxruntime.ai/docs/genai/tutorials/finetune.html
Is this planned?
# Describe scenario use case
## 1. Dynamic Model Switching in the Browser
Allow real-time switching between different fine-tuned behaviors: sentiment analysis, instruction tuning, role personas, or domain specializations by applying different LoRa adapters to a single base ONNX model. This avoids reloading or duplicating the entire model graph.
## 2. Reduced Memory Footprint for Multi-Model Applications
Loading multiple fine-tuned ONNX models currently requires storing several large model files in memory.
With LoRa adapter support, only **one base model** needs to stay loaded, while small LoRa adapters are applied at runtime. This significantly reduces RAM usage in:
- Browser environments
- Mobile or low-memory devices
- Multi-model applications
- Tooling that frequently swaps between fine-tuned variants
This solves the problem of needing multiple full ONNX models to run different fine-tune behaviors.
## 3. Faster Loading and Improved User Experience
LoRa adapters are small, enabling near-instant switching between fine-tuned behaviors without downloading or initializing an entire model again.
This improves UX in:
- Web-based LLM applications
- Interactive tools
- Apps where tasks change frequently (e.g., summarize → classify → generate)
This results in faster startup, reduced bandwidth use, and smoother runtime transitions.
## 4. Lower Bandwidth for Edge Devices
Users on limited networks or mobile connections benefit greatly from downloading small LoRa adapters instead of full multi-GB ONNX models.
Supporting LoRa adapters in onnxruntime-web:
- Cuts bandwidth consumption
- Reduces CDN/storage costs
- Enables on-device personalization without heavy downloads
- Makes large LLMs feasible in constrained environments
This is especially valuable for browser-based AI on edge hardware or mobile web apps.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ONNX Runtime GenAI fine-tuning tutorial linked in the issue and the onnxruntime-web implementation. Determine the browser-facing loading and inference entry points needed for .onnx_adapter files; done means a base ONNX model can load and use LoRa adapters at runtime.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning
- Domain
- machine-learning, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100