huggingface / huggingface/llm-ls

Deepseek Coder not working

Open
#92 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
881
Forks
71
Avg merge
33m
Merged PRs (30d)
1

Description

When trying to use deepseek coder (via ollama) and its tokenizer and tokens for fim, the result seems completely irrelevant (or, maybe, cut off). However, when using the prompt I would expect to go to the model directly in the ollama, everything works fine:

![image](https://github.com/huggingface/llm-ls/assets/63148080/15ebb241-af73-4362-b014-323cedd78968)

vs

![image](https://github.com/huggingface/llm-ls/assets/63148080/0aaad2f0-208d-42fb-a19d-9f2c442562d3)

Here is my config for llm.nvim:

```lua
require("llm").setup({
model = "deepseek-coder:1.3b-base",
enable_suggestions_on_startup = true,
accept_keymap = "",
dismiss_keymap = "",
tokens_to_clear = {
"<|endoftext|>",
},
fim = {
enabled = true,
prefix = "<|fim▁begin|>",
middle = "<|fim▁hole|>",
suffix = "<|fim▁end|>"
},
backend = "ollama",
debounce_ms = 0,
url = "http://localhost:11434/api/generate",
context_window = 240,
-- cf https://github.com/ollama/ollama/blob/main/docs/api.md#parameters
request_body = {
-- Modelfile options for the model you use
options = {
num_predict = 4,
temperature = 0.2,
top_p = 0.95,
},
},
lsp = {
bin_path = vim.api.nvim_call_function("stdpath", { "data" }) .. "/mason/bin/llm-ls",
},
tokenizer = {
repository = "deepseek-ai/deepseek-vl-1.3b-base", -- not working for some reason
},
})
```

I believe it is a problem with how llm-ls handles it, but if I am wrong, I will open an issue on the llm.nvim github

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.