huggingface / huggingface/llm-ls
Deepseek Coder not working
- Dominant language
- Rust
- Stars
- 881
- Forks
- 71
- Avg merge
- 33m
- Merged PRs (30d)
- 1
Description
When trying to use deepseek coder (via ollama) and its tokenizer and tokens for fim, the result seems completely irrelevant (or, maybe, cut off). However, when using the prompt I would expect to go to the model directly in the ollama, everything works fine:

vs

Here is my config for llm.nvim:
```lua
require("llm").setup({
model = "deepseek-coder:1.3b-base",
enable_suggestions_on_startup = true,
accept_keymap = "",
dismiss_keymap = "",
tokens_to_clear = {
"<|endoftext|>",
},
fim = {
enabled = true,
prefix = "<|fim▁begin|>",
middle = "<|fim▁hole|>",
suffix = "<|fim▁end|>"
},
backend = "ollama",
debounce_ms = 0,
url = "http://localhost:11434/api/generate",
context_window = 240,
-- cf https://github.com/ollama/ollama/blob/main/docs/api.md#parameters
request_body = {
-- Modelfile options for the model you use
options = {
num_predict = 4,
temperature = 0.2,
top_p = 0.95,
},
},
lsp = {
bin_path = vim.api.nvim_call_function("stdpath", { "data" }) .. "/mason/bin/llm-ls",
},
tokenizer = {
repository = "deepseek-ai/deepseek-vl-1.3b-base", -- not working for some reason
},
})
```
I believe it is a problem with how llm-ls handles it, but if I am wrong, I will open an issue on the llm.nvim github
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.