lm-sys / lm-sys/FastChat

Pydantic version conflict with vLLM

Open
#3,070 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

In short, vLLM depends on pydantic >= 2,
```
pydantic >= 2.0 # Required for OpenAI server.
```
on the other hand, `fastchat/serve/openai_api_server.py` depends on v1.x
```python
try:
from pydantic.v1 import BaseSettings
except ImportError:
from pydantic import BaseSettings
```

I installed the dependencies by `pip install fschat[model_worker] vllm`, it causes error at runtime:
```
Traceback (most recent call last):

File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main

return _run_code(code, main_globals, None,

File "/usr/lib/python3.10/runpy.py", line 86, in _run_code

exec(code, run_globals)

File "/usr/local/lib/python3.10/dist-packages/fastchat/serve/openai_api_server.py", line 24, in

from pydantic import BaseSettings

File "/usr/local/lib/python3.10/dist-packages/pydantic/__init__.py", line 374, in __getattr__

return _getattr_migration(attr_name)

File "/usr/local/lib/python3.10/dist-packages/pydantic/_migration.py", line 296, in wrapper

raise PydanticImportError(

pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.6/migration/#basesettings-has-moved-to-pydantic-settings for more details.

```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with fastchat/serve/openai_api_server.py and reproduce the failure using the reported `pip install fschat[model_worker] vllm` command. Trace the BaseSettings import and dependency versions; done means the OpenAI API server starts successfully with vLLM installed without the Pydantic import error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.