Enhancement: bundle BGE-base / BGE-small alongside all-MiniLM (measured +0.08 nDCG@10 on BEIR NFCorpus)

Summary

Bundle BGE-base-en-v1.5 and BGE-small-en-v1.5 alongside the current all-MiniLM-L6-v2 so downstream Node packages can get SOTA dense retrieval (~+0.08 nDCG@10 on BEIR NFCorpus) without the fragile @xenova/transformers → sharp → libvips dependency chain.

Measured value

While building a BEIR public-benchmark harness for ruvnet/ruflo (see ADR-085), we measured the difference between embedder choices on BEIR NFCorpus (N=323 test queries):

Embedder	nDCG@10	Rank vs 11 published baselines	Pipeline
(silent hash fallback — the bug)	0.262	11/11	xenova → sharp ↗ broken
all-MiniLM-L6-v2 (ruvector currently bundled)	~0.27 (published)	~10/11	bundled, works
BGE-base-en-v1.5 (110M)	0.352	2/11	xenova direct API (no sharp)
BGE-large-en-v1.5 (335M, published)	0.380	1/11	—

BGE-base lifts nDCG@10 by ~0.08 over all-MiniLM — that's the difference between rank-10 and rank-2 on a published BEIR leaderboard.

Why bundling matters (vs "just use xenova directly")

We currently load BGE-base via @xenova/transformers AutoTokenizer + AutoModel:

const mod = await import('@xenova/transformers');
const tokenizer = await mod.AutoTokenizer.from_pretrained('Xenova/bge-base-en-v1.5', { quantized: true });
const model = await mod.AutoModel.from_pretrained('Xenova/bge-base-en-v1.5', { quantized: true });

This works if @xenova/transformers initialises. On darwin-arm64 with pnpm, the chain breaks because:

• @xenova/transformers depends on sharp for image preprocessing
• sharp requires libvips shared libraries
• pnpm doesn't always link the per-arch .node binding correctly (sharp-darwin-arm64v8.node was missing in our install)

Text bi-encoders don't need image preprocessing, but the dep tree pulls it in anyway. ruvector already bundles all-MiniLM-L6-v2 without sharp — proving this is solvable. Adding BGE just extends that pattern to a stronger model.

The downstream benefit: ruflo's neural-tools.ts could replace the 3-tier xenova/agentic-flow cascade with a single tier-0 call to ruvector.embed(text) and get real, fast, contention-free embeddings every time. No hash-fallback bug. No platform-specific binding hell.

Proposed API

// Today
initOnnxEmbedder()                  // loads all-MiniLM-L6-v2
getOptimizedOnnxEmbedder()          // 384-dim

// Proposed — opt-in model selection
initOnnxEmbedder({ model: 'bge-base' })   // 768-dim
initOnnxEmbedder({ model: 'bge-small' })  // 384-dim, faster
initOnnxEmbedder({ model: 'minilm' })     // legacy default

getOptimizedOnnxEmbedder()          // returns the active embedder
// .embed() / .embedBatch() / .getDimension() / .modelName()

Backward compat: initOnnxEmbedder() with no args keeps loading all-MiniLM-L6-v2.

Size budget

Model	Int8 ONNX	Tokenizer	Total per model
all-MiniLM-L6-v2 (current)	~22 MB	~0.5 MB	~22 MB
bge-small-en-v1.5	~33 MB	~0.5 MB	~33 MB
bge-base-en-v1.5	~110 MB	~0.5 MB	~110 MB
bge-large-en-v1.5	~440 MB	~0.5 MB	~440 MB

bge-base is the sweet spot (top-2 on NFCorpus at 1/3 the size of bge-large). bge-small is the "compatible-with-MiniLM-perf-budget" option.

If npm package size is a concern, an alternative is lazy download from huggingface on first init — but that re-introduces the network dependency.

Related

• ruvector issue: API contract issues in ONNX embedder (filed separately)
• ruflo: ADR-085 (BEIR harness), ADR-086 (silent-fallback bug story)
• BAAI BGE: https://github.com/FlagOpen/FlagEmbedding
• Xenova quantised mirrors: https://huggingface.co/Xenova/bge-base-en-v1.5

Happy to PR — start a thread if useful.

Start by tracing the existing initOnnxEmbedder() and getOptimizedOnnxEmbedder() entry points, then review the related API contract issue and ADR-085/ADR-086. Done means bundling the proposed BGE models with opt-in model selection while preserving the no-argument all-MiniLM default and the embedder methods described in the issue.