huggingface / huggingface/funes

Feature request: Support multilingual embedding models (non-English sessions)

Open
#144 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
419
Forks
32
Avg merge
8h 49m
Merged PRs (30d)
20

Description

Thanks a lot for open-sourcing funes — the "raw evidence with provenance" design is exactly what's been missing in this space.

I noticed the default embedding model is pinned to BAAI/bge-small-en-v1.5, while my sessions are mostly in Chinese, so retrieval quality is a concern for me.

I understand that pinning the embedding model ensures a consistent vector space across machines. Given that design, is there any plan to let users choose (or override) the embedding model locally — for example, via a flag like --embedding-model, with the model identity stamped into the memory as it is now?

Alternatively, would you consider supporting a multilingual model (e.g., bge-m3 or a multilingual-e5 variant) as the default or as an official alternative? Happy to hear your thoughts.

Contributor guide

Open the contributing guide

Research direction

Start by locating the current default BAAI/bge-small-en-v1.5 configuration and where the model identity is stamped into memory. Clarify whether the intended outcome is a --embedding-model override, an official multilingual alternative, or both, then define acceptance criteria for model selection and persisted identity.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.