huggingface / huggingface/llm-vscode

vllm localhost fails to accept json

Open
#147 1 comment 1 reaction 0 assignees View on GitHub
stale
Dominant language
TypeScript
Stars
1.3k
Forks
142
Avg merge
3m
Merged PRs (30d)
2

Description

the openai format provided has the following format for the body:
{
"model": "model_id",
"parameters": {
"max_new_tokens": 60,
...
},
"prompt":"text"
}

however the accepted format in VLLM does not have a parameters field. it should be of the format:
{
"model": "model_id",
"max_new_tokens": 60,
...
"prompt":"text"
}

When you try to set the request body and don't include the parameters as a field then it is automatically added.
This then throws a 400 request error.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by tracing where localhost vLLM/OpenAI-compatible request bodies are assembled, then verify the request shape and its 400-response behavior; done means an existing parameters field is preserved and is not added automatically.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.