lm-sys / lm-sys/FastChat

sglang_work not working on sglang v0.2.13

Open
#3,498 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
No merged PRs in 30d

Description

image Error in sys.excepthook: ubuntu-xllm-stream-1-1 | Original exception was: ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | Traceback (most recent call last): ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | return _run_code(code, main_globals, None, ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | File "/usr/lib/python3.10/runpy.py", line 86, in _run_code ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | exec(code, run_globals) ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/fastchat/serve/sglang_worker.py", line 290, in ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | runtime = sgl.Runtime( ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/sglang/api.py", line 40, in Runtime ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | return Runtime(*args, **kwargs) ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/sglang/srt/server.py", line 534, in __init__ ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | raise RuntimeError( ubuntu-xllm-stream-1-1 | 2024-08-27 12:54:55 | ERROR | stderr | RuntimeError: Initialization failed. Please see the error messages above. @merrymercy

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in fastchat/serve/sglang_worker.py around line 290, then inspect the sglang Runtime initialization traceback shown in the report. Reproduce the failure with sglang v0.2.13 and identify the earlier error messages referenced by RuntimeError. Done means the sglang worker initializes successfully instead of terminating with this error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.