huggingface / huggingface/llm-vscode
How to generate the response from locally hosted end point in vscode?
- Dominant language
- TypeScript
- Stars
- 1.3k
- Forks
- 142
- Avg merge
- 3m
- Merged PRs (30d)
- 2
Description
Hi,
I managed to plug the llm-vcode extension to point to the locally running endpoint. Now when I am selected the content like as below:
# function to sum 2 numbers in python
then Cmd+shif+a > llm: show code attribution
My local endpoint invokes and give the relevant response as well in below format
`{
"details": {
"best_of_sequences": [
{
"finish_reason": "length",
"generated_text": "test",
"generated_tokens": 1,
"prefill": [
{
"id": 0,
"logprob": -0.34,
"text": "test"
}
],
"seed": 42,
"tokens": [
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
],
"top_tokens": [
[
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
]
]
}
],
"finish_reason": "length",
"generated_tokens": 1,
"prefill": [
{
"id": 0,
"logprob": -0.34,
"text": "test"
}
],
"seed": 42,
"tokens": [
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
],
"top_tokens": [
[
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
]
]
},
"generated_text": "test"
}`
"generated_text": value is replaced with actual response with python sum function
After 200, I can see the anything related to generated code in vscode.
Please suggest to how to I can get generated response in vscode itself.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.