AnswerDotAI / AnswerDotAI/RAGatouille

Can anyone explain how exactly the reranker is calculating the score?

Open
#197 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
4k
Forks
276
PR merge metrics
No merged PRs in 30d

Description

Could someone explain how the reranker calculates its scores? I'm observing scores such as 15.67 and 12.33, which are unexpected because I anticipated scores like 97 or 96. I use LangChain's `get_relevant_docs `to retrieve documents with relevance scores such as 98 and 96, and then send these snippets to the ColBERT reranker for reranking. The reranker then returns scores like 15.67 and 12.33. Should I consider a higher reranker score as indicative of a better snippet? Additionally, I would like to understand the technique used by the reranker to compute these scores.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the ColBERT reranker's scoring entry point and comparing its output with LangChain's get_relevant_docs results. Document how the reranker computes and orders scores, whether higher scores are better, and why its scale differs from LangChain's relevance scores.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, search
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.