AnswerDotAI / AnswerDotAI/RAGatouille
Save the ColBERT encodings to disk.
- Dominant language
- Python
- Stars
- 4k
- Forks
- 276
- PR merge metrics
- No merged PRs in 30d
Description
I have a use case where I run ColBERT on CPU on a couple thousand documents. For this I don't use PLAID but the `encode` and `search_encoded_docs` methods and the search works fast enough, the problem is that encoding all these documents on CPU takes time and I don't want to encode everything everytime I deploy the model so I developed a way for saving and loading these encodings:
https://github.com/ChatFAQ/ChatFAQ/blob/cc19e4b85198062888d6320e59276db31461f4e9/chat_rag/chat_rag/retrievers/colbert_retriever.py#L163
If interested I could improve and integrate this into the `RAGPretrainedModel` or `ColBERT` classes and make a PR.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the referenced chat_rag/retrievers/colbert_retriever.py implementation around line 163, then inspect the encode and search_encoded_docs methods in RAGPretrainedModel or ColBERT. The issue is complete when encodings can be saved and loaded for later searches without re-encoding the documents.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100