[Feature]: Add Mean Reciprocal Rank (MRR) metric to rec_sys
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
🚀 Feature
MRR (Mean Reciprocal Rank) is a widely used metric for evaluating retrieval systems, recommendation systems, and RAG pipelines. Currently Ignite has HitRate in rec_sys but MRR is missing.
Steps to reproduce :
Add a MRR metric to ignite/metrics/rec_sys/ similar to the existing HitRate implementation. MRR measures the average of
1/rank of the first relevant item across all queries, which is particularly useful for evaluating retrieval quality in RAG systems.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in ignite/metrics/rec_sys/ by reading the existing HitRate implementation and its surrounding metric conventions. Add the MRR metric so it reports the average reciprocal rank of the first relevant item across queries, and verify its behavior against the project’s existing rec_sys metric checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100