huggingface / huggingface/llm-vscode
Issue with Self-Hosted LLM Integration Using vLLM
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 142
- Avg merge
- 3m
- Merged PRs (30d)
- 2
Description
Hi,
I’m trying to integrate a self-hosted LLM with the extension, which uses vLLM (supports the OpenAI API standard). I've configured the HTTP endpoint and model name, but I'm unsure where to include the API key.
I tried to add headers to the json settings, but I'm facing the following error when the code completion request is sent:
`serde json error: data did not match any variant of untagged enum OpenAIAPIResponse`
Could you guide me on the correct approach for passing the API key?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
No file, test, or entry point is identified in the issue. Start by tracing how the HTTP endpoint, model name, and request headers are configured for the vLLM OpenAI-compatible service, then reproduce the code-completion request and inspect the serde json error. Done means the API key can be passed through the supported configuration and the response is handled without the OpenAIAPIResponse parsing error.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, api, authentication
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100