AnswerDotAI / AnswerDotAI/RAGatouille

Add support for 'quantize_embeddings'

Open
#181 1 comment 5 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
4k
Forks
276
PR merge metrics
No merged PRs in 30d

Description

I'm looking for quantize the embeddings to speed up the process of indexing / searching / etc.

For example in `sentence_transformers` there is [quantize_embeddings](https://sbert.net/docs/package_reference/quantization.html#sentence_transformers.quantization.quantize_embeddings):
```python
binary_embeddings = quantize_embeddings(embeddings, precision="binary" | "int8")
```

https://sbert.net/examples/applications/embedding-quantization/README.html

Is anything similar already available? Or is it possible to add it for the usage of ColBERT in RAGatouille?

Contributor guide

No contributing guide indexed for this repository

Research direction

No repository files, tests, or entry points are named. Start by tracing how RAGatouille handles embeddings during indexing and searching, then compare those paths with the linked sentence-transformers quantization documentation. Done means an agreed design and support for the requested embedding quantization modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.