microsoft/onnxruntime-genai

Generative AI extensions for onnxruntime

Stars: 1.1k
Forks: 354
Open beginner issues: 0
Indexed issues: 145
Avg merge: 2d 16h
Merged PRs (30d): 85

Dominant language: C++
License: MIT
Last GitHub push: Sep 18, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

145 open issues indexed

PagedAttention: CUDA illegal memory access on a full-context request when max_batch_size>=2 and max_scheduled_tokens=2048 Open

ep:CUDA

microsoft/onnxruntime-genai#2583 · 1 reaction · 2 assignees · Opened Sep 16, 2026
Any plans for a WebAssembly build? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#2536 · Opened Sep 5, 2026
Eliminate per-step allocations from Engine-hosted MTP chaining Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

microsoft/onnxruntime-genai#2517 · Opened Sep 2, 2026
[Feature request]: Add C++ multimodal benchmark (VLM perf testing without Python) Open

platform:mobile

microsoft/onnxruntime-genai#2373 · 1 reaction · 2 assignees · Opened Aug 5, 2026
[CUDA] Deterministic token-doubling / corrupted output on H200 (Hopper sm_90); CPU clean Open

ep:CUDA

Difficulty 5/5 Over a week Newbie friendliness 32/100

microsoft/onnxruntime-genai#2291 · 3 comments · Opened Jul 12, 2026
Gemma-4 E2B multimodal (ORT-GenAI) eval vs HF PyTorch: quality-parity findings + per_layer_inputs release note Open

microsoft/onnxruntime-genai#2288 · 1 assignee · Opened Jul 9, 2026
Running in nvidia agx orin jetpack7.2 Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#2283 · Opened Jul 9, 2026
Gemma3ForCausalLM ONNX export produces garbage output for certain prompts (Persian, and any prompt triggering Gemma3's known "massive activation" pattern) - divergence traced to layer 0 (sliding-attention) output Open

microsoft/onnxruntime-genai#2277 · 1 comment · 1 reaction · 2 assignees · Opened Jul 7, 2026
Crash on Android: Ort::InitApi() aborts because libonnxruntime.so resolves to the Linux build, not the Android AAR Open

platform:mobile

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/onnxruntime-genai#2243 · Opened Jun 26, 2026
Building with WinML on Windows Open

platform:windows

Difficulty 3/5 1-2 days Newbie friendliness 45/100

microsoft/onnxruntime-genai#2231 · 6 comments · Opened Jun 16, 2026
ModelBuilder support use_cache=False and hidden_states_layers Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#2217 · Opened Jun 11, 2026
Data-dependent bidirectional mask (Gemma-4 vision-block) forces standard Attention over GQA — split prefill/decode decoder graphs? Open

microsoft/onnxruntime-genai#2204 · 2 comments · 2 assignees · Opened Jun 9, 2026
Support suppress_tokens / begin_suppress_tokens (HF generation parity) Open

microsoft/onnxruntime-genai#2201 · 1 comment · 1 reaction · 2 assignees · Opened Jun 8, 2026
Feature request : Support for SmolVLM2 Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

microsoft/onnxruntime-genai#2197 · Opened Jun 8, 2026
Gemma4 Per-Layer Embedding: Architecture Analysis & Alternatives Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

microsoft/onnxruntime-genai#2149 · 2 comments · Opened May 8, 2026
Generator.rewind_to(0) fails for multimodal models (gemma4, phi4mm) Open

Difficulty 4/5 3-5 days Newbie friendliness 58/100

microsoft/onnxruntime-genai#2140 · Opened May 8, 2026
Copilot design of Pipeline-as-Config Open

api:java ep:DML platform:mobile

microsoft/onnxruntime-genai#2114 · 5 comments · 1 assignee · Opened May 2, 2026
Question: Can onnxruntime-genai.so be used in vanilla onnxruntime as a custom ops library? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#2080 · Opened Apr 11, 2026
Feature request: add ONNX Runtime GenAI support for Granite Speech models (e.g. granite-4.0-1b-speech-ONNX) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

microsoft/onnxruntime-genai#2072 · Opened Apr 7, 2026
cannot build Qwen/Qwen3-14B on machine with 64GB RAM Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

microsoft/onnxruntime-genai#2047 · Opened Mar 26, 2026
"profiling_level": "detailed" is not working for multi models qnn npu Open

platform:mobile

microsoft/onnxruntime-genai#2037 · 1 reaction · 2 assignees · Opened Mar 19, 2026
"enable_profiling" in session_options is not working for dml Open

ep:DML platform:mobile

Difficulty 4/5 3-5 days Newbie friendliness 28/100

microsoft/onnxruntime-genai#2031 · Opened Mar 17, 2026
[Java] `onnxruntime-genai-cuda.dll` missing from Java JAR and extraction logic in CUDA builds Open

api:java ep:CUDA

Difficulty 3/5 1-2 days Newbie friendliness 55/100

microsoft/onnxruntime-genai#2028 · 1 comment · Opened Mar 15, 2026
[Bug?] library crash at runtime when loading the model Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/onnxruntime-genai#2023 · 2 comments · Opened Mar 13, 2026
Feature Request: Support for Qwen3.5 Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

microsoft/onnxruntime-genai#2016 · 5 reactions · Opened Mar 10, 2026
[Privacy/Thermal] WorkloadsSessionHost silently consumes 90%+ GPU on Copilot+ PCs without user opt-in Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#2013 · 2 comments · Opened Mar 7, 2026
Using Onnxruntime Custom Execution Provider for ORT-GenAI Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#2006 · Opened Mar 4, 2026
Config::overlay() on pipeline session_options causes "File doesn't exist" on Model load Open

Difficulty 3/5 1-2 days Newbie friendliness 55/100

microsoft/onnxruntime-genai#2004 · Opened Mar 4, 2026
Document the config format Open

Difficulty 2/5 1-3 hours Newbie friendliness 30/100

microsoft/onnxruntime-genai#1990 · Opened Feb 24, 2026
[copilot] Qwen3-VL Support Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

microsoft/onnxruntime-genai#1989 · 1 comment · 7 reactions · Opened Feb 24, 2026
Unsupported tokenizer class for microsoft/OptiMind-SFT Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1987 · 2 comments · Opened Feb 23, 2026
.net 6 target for manged GenAI Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#1982 · 3 comments · Opened Feb 16, 2026
Feature Request: Support for Qwen3-ASR Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#1968 · 1 reaction · Opened Feb 6, 2026
bug: backend_type is ignored for QNN Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

microsoft/onnxruntime-genai#1963 · Opened Feb 3, 2026
Long context(input prompt length 4096 + generative output length 6000) may encounter a correctness issue for phi model in genai? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1932 · 1 comment · Opened Dec 23, 2025
Long context(GetSequenceLength >= 4097 in generator.cpp) encountered a perf regression in genai Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1910 · 5 comments · Opened Dec 9, 2025
Crash when loading a model the 2nd time after recreating OgaHandle Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1896 · Opened Nov 26, 2025
Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException:“Cuda interface not available: Failed to load library: Error loading "onnxruntime-genai-cuda.dll" which is missing. (Error 126)” Open

ep:CUDA

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/onnxruntime-genai#1887 · Opened Nov 22, 2025
DataContent does not work with images or audio Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1879 · Opened Nov 17, 2025
Unable to build python wheel with cuda 13 Open

ep:CUDA

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1877 · 1 comment · Opened Nov 15, 2025
OnnxRuntimeGenAIChatClient in WINUI3 S4 Hibernation Scenario: Model Residue When Not Disposed, Crash on Dispose() Call Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1836 · Opened Oct 21, 2025
Description of onnxRuntimeGenAI.QNN Long-Term Inference Output Anomaly Issue Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

microsoft/onnxruntime-genai#1831 · Opened Oct 16, 2025
onnxRuntimeGenAI.QNN Long-Term Inference Crash and Memory Anomalies Issue Report Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

microsoft/onnxruntime-genai#1830 · Opened Oct 16, 2025
Better error message for when model.type is incorrect Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1825 · Opened Oct 14, 2025
Lack of Selective Adapter Loading in Phi-4-multimodal-instruct-onnx Open

msedge

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1821 · Opened Oct 13, 2025
Onnx Runtime Model Dispose Crash When Multiple Models are Loaded Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

microsoft/onnxruntime-genai#1796 · 1 comment · Opened Sep 29, 2025
Support for dynamic int8 quantization Open

quantization

Difficulty 4/5 3-5 days Newbie friendliness 42/100

microsoft/onnxruntime-genai#1792 · Opened Sep 25, 2025
Symbol Publishing Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

microsoft/onnxruntime-genai#1783 · 1 comment · Opened Sep 19, 2025
[Multimodal] Allow raw bitmap images to be used for OgaProcessorProcessImages Open

msedge

Difficulty 4/5 3-5 days Newbie friendliness 45/100

microsoft/onnxruntime-genai#1770 · Opened Sep 17, 2025
Lack of requirements for builder.py Open

ep:DML

Difficulty 2/5 1-3 hours Newbie friendliness 50/100

microsoft/onnxruntime-genai#1759 · Opened Sep 15, 2025

A short digest of beginner-friendly GitHub issues.