lm-sys / lm-sys/RouteLLM

can not initializing the RouteLLM controller

Open
#21 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.5k
Forks
433
PR merge metrics
No merged PRs in 30d

Description

when i initializing the RouteLLM controller with demo, i meet the errors below:

```
import os
from routellm.controller import Controller

os.environ["OPENAI_API_KEY"] = "sk-xxxxxxx"
os.environ["ANYSCALE_API_KEY"] = "esecret_xxxxx"

client = Controller(
routers=["mf"],
strong_model="gpt-4-1106-preview",
weak_model="anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1",
)

response = client.chat.completions.create(
model="router-mf-0.11593",
messages=[
{"role": "user", "content": "Hello!"}
]
)
```

```
Traceback (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/openai_server.py", line 22, in
from routellm.controller import Controller, RoutingError
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/controller.py", line 10, in
from routellm.routers.routers import ROUTER_CLS
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/routers.py", line 17, in
from routellm.routers.matrix_factorization.model import MODEL_IDS, MFModel
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/matrix_factorization/model.py", line 4, in
from routellm.routers.similarity_weighted.utils import OPENAI_CLIENT
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/routellm/routers/similarity_weighted/utils.py", line 8, in
from sklearn.linear_model import LogisticRegression
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/__init__.py", line 84, in
from .base import clone
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/base.py", line 19, in
from .utils._estimator_html_repr import _HTMLDocumentationLinkMixin, estimator_html_repr
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/__init__.py", line 11, in
from ._chunking import gen_batches, gen_even_slices
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_chunking.py", line 8, in
from ._param_validation import Interval, validate_params
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_param_validation.py", line 14, in
from .validation import _is_arraylike_not_scalar
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/validation.py", line 26, in
from ..utils._array_api import _asarray_with_order, _is_numpy_namespace, get_namespace
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/_array_api.py", line 11, in
from .fixes import parse_version
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/sklearn/utils/fixes.py", line 20, in
import scipy.stats
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/__init__.py", line 606, in
from ._stats_py import *
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/_stats_py.py", line 49, in
from . import distributions
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/distributions.py", line 11, in
from . import _discrete_distns
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/stats/_discrete_distns.py", line 10, in
from scipy.interpolate import interp1d
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/__init__.py", line 167, in
from ._interpolate import *
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_interpolate.py", line 14, in
from . import _fitpack_py
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_fitpack_py.py", line 8, in
from ._fitpack_impl import bisplrep, bisplev, dblint # noqa: F401
File "/data/RouteLLM/.routellm/lib/python3.9/site-packages/scipy/interpolate/_fitpack_impl.py", line 103, in
'iwrk': array([], dfitpack_int), 'u': array([], float),
TypeError
```

i use python 3.9.5 and install routellm with

```
python3 -m venv .routellm
source .routellm/bin/activate

pip install "routellm[serve,eval]"
```

pls help to find the problem, ths!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Python 3.9.5 installation and follow the import chain from routellm/controller.py through routellm/routers/routers.py, routellm/routers/matrix_factorization/model.py, and routellm/routers/similarity_weighted/utils.py. Done means the documented routellm[serve,eval] setup can import Controller without the reported scipy TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
backend, developer-experience
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.