software-mansion / software-mansion/react-native-executorch
[RNE Rewrite] Add LFM2.5 ColBERT (late-interaction multi-vector) embedding pipeline
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.7k
- Forks
- 96
- Avg merge
- 20h 51m
- Merged PRs (30d)
- 59
Description
Summary
Add a LFM2.5-ColBERT-350M late-interaction (multi-vector) embedding pipeline under the new flow. Follow-up to the single-vector embeddings work in #1292 / #1247.
Why it isn't covered by the text embedder
ColBERT is a late-interaction model: forward returns a per-token matrix [numTokens, 128] (Linear 1024→128 + per-token L2-norm), not a single pooled vector. Relevance is scored with MaxSim (for each query token, the max cosine against all document tokens, summed) rather than a single cosine. So createTextEmbedder / useTextEmbedder cannot represent it — it needs its own pipeline.
The export scripts already exist (exporter_models/lfm2_5_colbert_350m, xnnpack + mlx) and an HF repo (react-native-executorch-lfm2.5-colbert-350m, currently only v0.9.0 / old flow).
Scope
Export (export-scripts, lfm2_5_colbert_350m)
- Port both backends to the new flow (dynamic seq +
get_dynamic_dims_forwardcompanion), matching the embedding export - MLX must be built with the
software-mansion-labs/executorchfork + MLX-native quantization — a pip/torchao-built MLX delegate fails at execute withError::InvalidProgram(same root cause as the #1292 MLX fix; the MLX backend is version-locked to the runtimeMLXBackend.xcframework) - Publish xnnpack + mlx to HF
react-native-executorch-lfm2.5-colbert-350mv0.10.0
Library
- New task
createColbertEmbedder+useColbertEmbedderhook returning per-token embeddings[numTokens, 128] -
MaxSimlate-interaction scoring helper (query vs. document token matrices) -
models.tsentries (xnnpack 8da4w + mlx int4) + index exports;download-libsalready enables the MLX backend fortextEmbeddings
Demo (nlp)
- Late-interaction retrieval screen (or extend Text Embeddings) using MaxSim; MLX chip disabled on Android (MLX is iOS-only)
Notes
- The exporter NOTES document that XNNPACK lowering bakes the dynamic seq dim to static
[1,512]/[1,512,128]on the current ET build; decide between static + runtime padding vs. a build that preserves the dynamic seq (the embedding MLX variant shipped dynamic; both static+pad and dynamic were verified on device). - Related: standalone on-device vector store #1303.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the single-vector embedding work in #1292/#1247 and the exporter_models/lfm2_5_colbert_350m scripts first. Then trace models.ts, index exports, download-libs, and the nlp demo to understand the library and UI integration points. Done means both backends are published as v0.10.0, the ColBERT APIs and MaxSim helper work, and the demo handles MLX's iOS-only support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch, react-native, typescript
- Domain
- machine-learning, mobile
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100