APIError: Invalid response object from API: 'Internal Server Error' (HTTP response code was 500)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
python -m fastchat.serve.controller
python -m fastchat.serve.vllm_worker --model-path ....../Qwen2___5-7B-Instruct --trust-remote-code --dtype bfloat16
python -m fastchat.serve.openai_api_server --host localhost --port 8000
import openai
openai.api_key = "EMPTY"
openai.api_base = "http://localhost:8000/v1"
model = "Qwen2___5-7B-Instruct"
prompt = "Once upon a time"
# create a completion
completion = openai.Completion.create(model=model, prompt=prompt)
# # print the completion
# print(prompt + completion.choices[0].text)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report by starting the controller, vllm_worker, and openai_api_server with the commands shown, then run the provided openai.Completion.create example. Start by tracing the request through the openai_api_server entry point and capture the server traceback for the HTTP 500. Done means the request has a clear resolution and the API behavior is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100