huggingface / huggingface/transformers.js

ColBERT Wasm Demo

Open
#656 7 comments 5 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

It would be worth to provide an example of using ColBERT passage retrieval from a user query as browser execution.
A good example has been provide [here](https://colbert.aiserv.cloud/) for query-passage score interpretability

![Screenshot 2024-03-21 at 10 22 01](https://github.com/xenova/transformers.js/assets/163333/a7833dab-abbe-4454-9f4a-677996d91b88)

Specifically the Contextualised Highlights gives an important overview of the inner scoring at the token level, as well as the resulting MaxSim score.

**Motivation.**
Recently I was able to benchmark ColBERT for WASM cpu execution vs. WebGPU thanks to Xenova playground [here](https://huggingface.co/spaces/Xenova/webgpu-embedding-benchmark), ColBERT (with quantization eventually) performances in the browser are efficient enough to perform passage retrieval locally with WebGPU support and eventually fallback to cpu.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.