AnswerDotAI / AnswerDotAI/RAGatouille
Question: How to get score of ranked document?
- Dominant language
- Python
- Stars
- 4k
- Forks
- 276
- PR merge metrics
- No merged PRs in 30d
Description
Hey all, I'm using RAGatouille as a retriever for langchain like so
````Python
RAG = RAGPretrainedModel.from_index(".ragatouille/colbert/indexes/clr-docs")
retriever = RAG.as_langchain_retriever(k=10)
retrieval = RunnableParallel({
"context": retriever,
"question": RunnablePassthrough(),
})
rag_chain = (
retrieval
| RunnableParallel({
"answer": ANSWER_PROMPT | llm,
"docs": itemgetter("context")
})
)
````
I've traced down `as_langchain_retriever` but I could not find any helpful arguments.
How do I get the `score` for each retrieved doc so I can, i.e., calculate the auPRC score?
Thanks,
Bernhard.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue points to RAG.as_langchain_retriever(k=10) and a LangChain retrieval chain, but names no repository file or test. Start by tracing that entry point and inspect the returned document interface and existing retrieval examples. Done means establishing a documented way to obtain each retrieved document's score for auPRC calculation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100