huggingface / huggingface/llm-vscode

Weird behavior with "codellama/CodeLlama-13b-hf"

Open
#113 7 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
TypeScript
Stars
1.3k
Forks
142
Avg merge
3m
Merged PRs (30d)
2

Description

Suggestion is not being displayed when using CodeLlama. This is not the case with Starcoder, it shows the suggestion in the line that's triggered from.

Here are the attempts:

#### Requesting from Line 5, nothing gets displayed.
![image](https://github.com/huggingface/llm-vscode/assets/60504285/7cae0c3a-ca75-438b-baa8-23c501c9508a)

#### Requesting from Line 6 with a one space to the right, shows the suggestion.
![image](https://github.com/huggingface/llm-vscode/assets/60504285/dc16fc9e-8584-4ef5-9e90-a53f74963131)

### Settings
```json
{
"llm.fillInTheMiddle.enabled": true,
"llm.fillInTheMiddle.prefix": "

 ",

"llm.fillInTheMiddle.middle": " ",
"llm.fillInTheMiddle.suffix": " ",
"llm.temperature": 0.2,
"llm.contextWindow": 4096,
"llm.tokensToClear": [
""
],
"llm.tokenizer": {
"repository": "codellama/CodeLlama-13b-hf"
},
"llm.enableAutoSuggest": true,
"llm.maxNewTokens": 256,
"llm.configTemplate": "codellama/CodeLlama-13b-hf",
"llm.modelIdOrEndpoint": "codellama/CodeLlama-13b-hf"
}
```

### Example Code
```go
package main
import "fmt"

func main() {
// Cursor here
func() { messages <- "ping" }()
msg = <-messages
fmt.Println(msg)
}
```

### VSCode
Version: 1.85.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.