AnswerDotAI / AnswerDotAI/RAGatouille

Why is Searcher loading a second Checkpoint, i.e. the same model again?

Open
#175 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4k
Forks
276
PR merge metrics
No merged PRs in 30d

Description

When following the basic examples I noticed that the `Searcher` class when called through `RAG.search` will always load the pytorch model used in the index (e.g. `colbert-ir/colbertv2.0`) a second time, by calling `self.checkpoint = Checkpoint(self.checkpoint, colbert_config=self.config, verbose=self.verbose)`. Is there a specific reason for doing this? Couldn't it just use the `Checkpoint` that was previously loaded when the index was loaded, to save on the extra loading time and avoid double the memory use?

Steps to reproduce, based on the example [here](https://github.com/bclavie/RAGatouille/blob/main/examples/01-basic_indexing_and_search.ipynb)
```
# will load the model `colbert-ir/colbertv2.0` the first time
RAG = RAGPretrainedModel.from_index(".ragatouille/colbert/indexes/Miyazaki/")

# will load the model a second time
RAG.search(query="What animation studio did Miyazaki found?")
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the basic indexing and search example in examples/01-basic_indexing_and_search.ipynb, following RAGPretrainedModel.from_index and RAG.search into Searcher and its Checkpoint initialization. Confirm whether the model is loaded twice, then determine whether the existing Checkpoint can be reused; done means search works without the second model load and avoids the duplicate memory use.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.