AnswerDotAI / AnswerDotAI/RAGatouille
Why is Searcher loading a second Checkpoint, i.e. the same model again?
- Dominant language
- Python
- Stars
- 4k
- Forks
- 276
- PR merge metrics
- No merged PRs in 30d
Description
When following the basic examples I noticed that the `Searcher` class when called through `RAG.search` will always load the pytorch model used in the index (e.g. `colbert-ir/colbertv2.0`) a second time, by calling `self.checkpoint = Checkpoint(self.checkpoint, colbert_config=self.config, verbose=self.verbose)`. Is there a specific reason for doing this? Couldn't it just use the `Checkpoint` that was previously loaded when the index was loaded, to save on the extra loading time and avoid double the memory use?
Steps to reproduce, based on the example [here](https://github.com/bclavie/RAGatouille/blob/main/examples/01-basic_indexing_and_search.ipynb)
```
# will load the model `colbert-ir/colbertv2.0` the first time
RAG = RAGPretrainedModel.from_index(".ragatouille/colbert/indexes/Miyazaki/")
# will load the model a second time
RAG.search(query="What animation studio did Miyazaki found?")
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the basic indexing and search example in examples/01-basic_indexing_and_search.ipynb, following RAGPretrainedModel.from_index and RAG.search into Searcher and its Checkpoint initialization. Confirm whether the model is loaded twice, then determine whether the existing Checkpoint can be reused; done means search works without the second model load and avoids the duplicate memory use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100