Gradio web server requires model-worker deps
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I plan to make a lighter Docker image when only API-based inference is needed.
But torch is imported when getting conversation templates.
Do you have any opinion on how to separate this?
Thanks.
Here are the logs:
```py
fastchat-gradio-web-server-1 | Traceback (most recent call last):
fastchat-gradio-web-server-1 | File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
fastchat-gradio-web-server-1 | return _run_code(code, main_globals, None,
fastchat-gradio-web-server-1 | File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
fastchat-gradio-web-server-1 | exec(code, run_globals)
fastchat-gradio-web-server-1 | File "/app/fastchat/serve/gradio_web_server_multi.py", line 12, in
fastchat-gradio-web-server-1 | from fastchat.serve.gradio_block_arena_anony import (
fastchat-gradio-web-server-1 | File "/app/fastchat/serve/gradio_block_arena_anony.py", line 19, in
fastchat-gradio-web-server-1 | from fastchat.model.model_adapter import get_conversation_template
fastchat-gradio-web-server-1 | File "/app/fastchat/model/__init__.py", line 1, in
fastchat-gradio-web-server-1 | from fastchat.model.model_adapter import (
fastchat-gradio-web-server-1 | File "/app/fastchat/model/model_adapter.py", line 16, in
fastchat-gradio-web-server-1 | import torch
fastchat-gradio-web-server-1 | ModuleNotFoundError: No module named 'torch'
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fastchat/serve/gradio_web_server_multi.py and fastchat/serve/gradio_block_arena_anony.py, then trace the import path into fastchat/model/model_adapter.py where torch is imported. Run the Gradio web server using an image without torch and determine the dependency boundary needed for API-based inference. Done means the API-only server starts without requiring model-worker dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100