huggingface / huggingface/llm-ls

Can not using the ollama in docker container. ERROR: [LLM] http error

Open
#99 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
881
Forks
71
Avg merge
33m
Merged PRs (30d)
1

Description

It is a great plugin and I love it. But I found an error here.
```
[LLM] http error: error sending request for url (http://localhost:11434/api/generate): connection closed before message completed
```
Following the config in the readme.
```
{
"huggingface/llm.nvim",
opts = {
-- cf Setup
},
config = function()
local llm = require("llm")
llm.setup({
api_token = nil, -- cf Install paragraph
-- for ollama backend
backend = "ollama", -- backend ID, "huggingface" | "" | "openai" | "tgi"
model = "starcoder2:7b",
url = "http://localhost:11434/api/generate",
tokens_to_clear = { "<|endoftext|>" }, -- tokens to remove from the model's output
-- parameters that are added to the request body, values are arbitrary, you can set any field:value pair here it will be passed as is to the backend
request_body = {
parameters = {
max_new_tokens = 60,
temperature = 0.2,
top_p = 0.95,
},
},
-- set this if the model supports fill in the middle
fim = {
enabled = true,
prefix = "",
middle = "",
suffix = "",
},
debounce_ms = 150,
accept_keymap = "",
dismiss_keymap = "",
tls_skip_verify_insecure = false,
-- llm-ls configuration, cf llm-ls section
lsp = {
bin_path = nil,
host = nil,
port = nil,
version = "0.5.2",
},
tokenizer = {
repository = "bigcode/starcoder2-7b",

}, -- cf Tokenizer paragraph
-- tokenizer = nil, -- cf Tokenizer paragraph
context_window = 4096, -- max number of tokens for the context window
enable_suggestions_on_startup = true,
enable_suggestions_on_files = "*", -- pattern matching syntax to enable suggestions on specific files, either a string or a list of strings
})
end,
}
```
The MOST wirred thing is that I can curl the answer to the same model & api url, and my vscode continue plugin can communicate this is ollama which is running on a docker container but this plugin cannot!

Thank you for your time and reply!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported Ollama request with the configuration in the issue, then compare the plugin's request with the successful curl call to the Docker-hosted API. Done means the configured Ollama backend receives the request and returns a complete response without the connection-closed error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, ollama
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.