mudler / mudler/LocalRecall

Switching embedding model makes localrecall stop working with key error

Open
#26 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
974
Forks
121
Avg merge
5h 52m
Merged PRs (30d)
1

Description

I'm using LocalRecall from sources with the latest upstream 03f36d6 on Mac T8132 MacOS 15.5

It does work fine when using the default embedding model granite-embedding-107m-multilingual but as soon as I switch to another model it starts failing. I tried to use qwen3-embedding-8b and after re-running localrecall it stops working when uploading a file with the following error:

"ERROR","msg":"Failed to store file","!BADKEY":"failed to store file: couldn't add document '101': couldn't create embedding of document: error getting keys: error, status code: 500, status: 500 Internal Server Error, message: rpc error: code = InvalidArgument desc = Input content cannot be empty","source":{"filename":"/******/localrecall/routes.go","lineno":247}}

When searching the vector stores the same error gets thrown:

{"error":"Failed to search collection: couldn't create embedding of query: error getting keys: error, status code: 500, status: 500 Internal Server Error, message: rpc error: code = InvalidArgument desc = Input content cannot be empty"}

Afaik the qwen model should do the same as the granite one. I tried switching to it because the granite model produces very inaccurate results for me in German. I thought a larger model could probably do better finding the right embeddings to my queries. Regardless of that, I think localrecall should work with the qwen embedding model if it's available in localai so I opened this issue.

Switching back to granite model makes localrecall work fine.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at routes.go:247 and reproduce the failure by configuring qwen3-embedding-8b, then compare file upload and search behavior with the granite-embedding-107m-multilingual model. Trace the embedding request until the LocalAI error reports empty input; done means uploads and searches work with the alternate model without the key error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.