Design decision: pretrained (lattice-backed) embeddings — default-on or opt-in? (fix path for #655)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 603
- Avg merge
- 23h 32m
- Merged PRs (30d)
- 59
Description
Issue filed by an AI agent on behalf of @ohdearquant.
Context
#655 reports that @ruvector/ruvllm's embed()/similarity are non-discriminative — the
embedder is a random-init character-level matrix (examples/ruvLLM/src/embedding.rs), never
loaded from a checkpoint, so unrelated pairs often out-score paraphrases.
A fix is in progress that backs EmbeddingService with a real pretrained embedder —
ruvector_core::embeddings::LatticeEmbedding (already merged into ruvector-core behind the
lattice-embeddings feature; supports MiniLM, BGE-small/base/large, E5, Qwen3-Embedding). It's
API-compatible with the existing EmbeddingService surface, so napi.rs doesn't change.
The one real decision isn't code — it's distribution, and it's yours to make. Filing this to
ask before assuming.
The fork
A. Default-on. embed()/similarity produce real semantic embeddings out of the box —
fixes #655 for every user with zero opt-in.
Cost: a pretrained model (~90 MB MiniLM / ~130 MB BGE-small) becomes a runtime dependency of the
published package (download-on-first-use unless bundled); the feature build requires Rust ≥ 1.93
(lattice-embed 0.6, edition 2024); heavier build.
B. Opt-in (what the in-progress PR does today). Default build, footprint, and MSRV unchanged;
the real embedder is available when the lattice-embeddings feature is compiled in and a model is
selected (via config/env). A visibility flag (pretrained: bool on the stats) makes the
random-vs-pretrained state explicit so the #655 failure mode is never silent. Downside: users
hitting #655 have to opt in to get the fix.
If you want A (default-on), three sub-choices
- Default model — MiniLM (smaller, symmetric, solid STS/similarity) vs BGE-small (asymmetric
query/passage prefixing, stronger retrieval). - Bundle vs lazy-download — ship the weights inside the npm package (bigger install, works
offline immediately) vs fetch on firstembed()(small install, needs network on first run). - The random path — keep it as an explicitly-selected offline/no-download fallback, or drop
the "embedding" framing there entirely (per #655's own third suggestion).
Ask
Which do you want — A or B, and if A, your calls on the three sub-choices? The PR is wired for B
right now; flipping it to default-on (plus wiring the chosen model and bundle/lazy behavior) is a
small change once you decide. Not going to change the default footprint of your published package
without your say-so.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/ruvLLM/src/embedding.rs and the EmbeddingService surface, then review the lattice-embeddings feature in ruvector_core and the existing napi.rs boundary. Resolve whether pretrained embeddings are default-on or opt-in, including the model and bundle or lazy-download choices if applicable; done means the distribution decision is recorded for the in-progress PR.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100