adithya-s-k / adithya-s-k/VARAG
Add Support for Auto Retrival Evaluation of different RAG techniques
- 主要语言
- Python
- 星标
- 502
- 派生
- 48
- PR 合并指标
- 30 天内没有已合并 PR
描述
### Issue: Add `.evaluate` and `.generate_eval_dataset` Methods for RAG Pipelines
**Description:**
Enhance the RAG framework by implementing `.evaluate` and `.generate_eval_dataset` methods to enable easy and automated evaluation of different RAG pipelines.
**Requirements:**
1. **`.evaluate` Method:**
- Allow users to benchmark retrieval performance using standard metrics (e.g., Precision, Recall, F1).
- Support configuration for top-k retrieval and similarity scoring.
2. **`.generate_eval_dataset` Method:**
- Automatically create datasets for evaluation with annotated ground truth queries and responses.
- Include support for custom datasets.
3. **Integration:**
- Seamlessly integrate with existing RAG pipelines.
**Impact:**
Simplifies evaluation workflows and helps compare RAG pipeline performance effectively.
**Priority:** Medium
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。