huggingface / huggingface/llm-vscode

How to generate the response from locally hosted end point in vscode?

Open
#100 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
TypeScript
Stars
1.3k
Forks
142
Avg merge
3m
Merged PRs (30d)
2

Description

Hi,

I managed to plug the llm-vcode extension to point to the locally running endpoint. Now when I am selected the content like as below:
# function to sum 2 numbers in python
then Cmd+shif+a > llm: show code attribution
My local endpoint invokes and give the relevant response as well in below format

`{
"details": {
"best_of_sequences": [
{
"finish_reason": "length",
"generated_text": "test",
"generated_tokens": 1,
"prefill": [
{
"id": 0,
"logprob": -0.34,
"text": "test"
}
],
"seed": 42,
"tokens": [
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
],
"top_tokens": [
[
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
]
]
}
],
"finish_reason": "length",
"generated_tokens": 1,
"prefill": [
{
"id": 0,
"logprob": -0.34,
"text": "test"
}
],
"seed": 42,
"tokens": [
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
],
"top_tokens": [
[
{
"id": 0,
"logprob": -0.34,
"special": false,
"text": "test"
}
]
]
},
"generated_text": "test"
}`
"generated_text": value is replaced with actual response with python sum function

After 200, I can see the anything related to generated code in vscode.

Please suggest to how to I can get generated response in vscode itself.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.