adithya-s-k / adithya-s-k/VARAG
Add Support for Auto Retrival Evaluation of different RAG techniques
- Linguagem predominante
- Python
- Estrelas
- 502
- Forks
- 48
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
### Issue: Add `.evaluate` and `.generate_eval_dataset` Methods for RAG Pipelines
**Description:**
Enhance the RAG framework by implementing `.evaluate` and `.generate_eval_dataset` methods to enable easy and automated evaluation of different RAG pipelines.
**Requirements:**
1. **`.evaluate` Method:**
- Allow users to benchmark retrieval performance using standard metrics (e.g., Precision, Recall, F1).
- Support configuration for top-k retrieval and similarity scoring.
2. **`.generate_eval_dataset` Method:**
- Automatically create datasets for evaluation with annotated ground truth queries and responses.
- Include support for custom datasets.
3. **Integration:**
- Seamlessly integrate with existing RAG pipelines.
**Impact:**
Simplifies evaluation workflows and helps compare RAG pipeline performance effectively.
**Priority:** Medium
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.