lm-sys/RouteLLM

View on GitHub

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

Stars: 5.5k
Forks: 433
Open beginner issues: 0
Indexed issues: 29

Dominant language: Python
License: Apache-2.0
Last GitHub push: Aug 10, 2024
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

29 open issues indexed

Portable format for router benchmark results (EvalPort) — worth exporting alongside the plots? Open

Difficulty 3/5 1-2 days Newbie friendliness 54/100

lm-sys/RouteLLM#93 · Opened Aug 23, 2026
How can I use the different embedding model for mf router Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

lm-sys/RouteLLM#84 · 1 reaction · Opened Feb 1, 2026
Are training scripts available? Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

lm-sys/RouteLLM#82 · 3 reactions · Opened Sep 17, 2025
Routing to Local Models problem Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

lm-sys/RouteLLM#79 · 1 reaction · Opened Mar 5, 2025
Following demo it doesn't work Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

lm-sys/RouteLLM#78 · Opened Feb 16, 2025
Can you please add model info for routellm/mf_gpt4_augmented and routellm/bert_gpt4_augmented Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

lm-sys/RouteLLM#74 · Opened Dec 10, 2024
Can BERT router source code (esp. for training) be added to the repo? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

lm-sys/RouteLLM#67 · 3 comments · 4 reactions · Opened Nov 5, 2024
How to train a router? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#65 · Opened Oct 29, 2024
litellm.drop_params error when running the openapi server Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

lm-sys/RouteLLM#63 · 2 comments · Opened Oct 7, 2024
Ability to Incorporate Open Source Embeddings for Router to Remove OpenAI dependency Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

lm-sys/RouteLLM#62 · Opened Oct 7, 2024
Support providing separate API keys for strong and weak models Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

lm-sys/RouteLLM#56 · 1 comment · Opened Aug 18, 2024
Does routellm support routing queries to different models in OpenAI assistant? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#54 · 2 comments · Opened Aug 9, 2024
model directory naming conflict Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

lm-sys/RouteLLM#52 · Opened Aug 7, 2024
use different embedding model Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

lm-sys/RouteLLM#50 · 1 comment · Opened Aug 6, 2024
dep version not pinned Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

lm-sys/RouteLLM#49 · Opened Aug 6, 2024
unable to edit/build with pdm Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

lm-sys/RouteLLM#47 · Opened Aug 6, 2024
Server doesn't respond to CORS preflight requests Open

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

lm-sys/RouteLLM#43 · Opened Aug 6, 2024
Can we use the matrix factorization model locally by downloading it to our local? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#39 · 2 comments · Opened Jul 31, 2024
OpenAIError: The api_key client option must be set Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

lm-sys/RouteLLM#38 · 5 comments · Opened Jul 30, 2024
Gemini-pro from openrouter have context length of 91k but getting error Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

lm-sys/RouteLLM#34 · 1 comment · Opened Jul 25, 2024
Can I use Azure open AI Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#29 · 5 comments · Opened Jul 22, 2024
Langchain integration Open

Difficulty 3/5 1-2 days Newbie friendliness 32/100

lm-sys/RouteLLM#28 · 3 comments · 3 reactions · Opened Jul 22, 2024
Allow Separate Base URLs for Strong and Weak Models in Controller Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

lm-sys/RouteLLM#27 · 1 comment · 2 reactions · Opened Jul 18, 2024
Provide support for more than two models and provide a training guide. Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#23 · 5 comments · 3 reactions · Opened Jul 11, 2024
can not initializing the RouteLLM controller Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

lm-sys/RouteLLM#21 · 4 comments · Opened Jul 10, 2024
Support function calling? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

lm-sys/RouteLLM#20 · 9 comments · Opened Jul 10, 2024
OpenAI dependency Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

lm-sys/RouteLLM#19 · 1 comment · Opened Jul 9, 2024
can support ollama? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

lm-sys/RouteLLM#16 · 2 comments · Opened Jul 9, 2024
Matrix factorization router is bloated Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

lm-sys/RouteLLM#9 · 2 comments · Opened Jul 6, 2024

lm-sys/RouteLLM

Get new issues in your inbox