huggingface / huggingface/llm-vscode
Running llama-cpp-python OpenAI compatible server
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 142
- Avg merge
- 3m
- Merged PRs (30d)
- 2
Description
Requesting a little help here. Trying to test out copilot functionality with `llama-cpp-python` with this extension. Below is my configuration setting.
```bash
{
"[python]": {
"editor.formatOnType": true
},
"cmake.configureOnOpen": true,
"llm.backend": "openai",
"llm.configTemplate": "Custom",
"llm.url": "http://192.X.X.X:12080/v1/chat/completions",
"llm.fillInTheMiddle.enabled": false,
"llm.fillInTheMiddle.prefix": "
",
"llm.fillInTheMiddle.middle": " ",
"llm.fillInTheMiddle.suffix": " ",
"llm.requestBody": {
"parameters": {
"max_tokens": 60,
"temperature": 0.2,
"top_p": 0.95
}
},
"llm.contextWindow": 4096,
"llm.tokensToClear": [
""
],
"llm.tokenizer": null,
"llm.tlsSkipVerifyInsecure": true,
"llm.modelId": "",
}
```I am seeing there is inference going on the server:
So I am not entirely sure what I am missing. Additionally I am trying to see the extension logs.. for the worker calls. But I don't see anything. Would you be able to give any guidance or some step by step explanation on how this can be done.
Thank you so much
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.