huggingface / huggingface/llm-vscode
vllm localhost fails to accept json
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 142
- Avg merge
- 3m
- Merged PRs (30d)
- 2
Description
the openai format provided has the following format for the body:
{
"model": "model_id",
"parameters": {
"max_new_tokens": 60,
...
},
"prompt":"text"
}
however the accepted format in VLLM does not have a parameters field. it should be of the format:
{
"model": "model_id",
"max_new_tokens": 60,
...
"prompt":"text"
}
When you try to set the request body and don't include the parameters as a field then it is automatically added.
This then throws a 400 request error.
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by tracing where localhost vLLM/OpenAI-compatible request bodies are assembled, then verify the request shape and its 400-response behavior; done means an existing parameters field is preserved and is not added automatically.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100