adithya-s-k / adithya-s-k/VARAG

Add Support for Auto Retrival Evaluation of different RAG techniques

未关闭
#5 0 条评论 0 个 reaction 已指派 1 人 已被 @adithya-s-k 认领 在 GitHub 查看
主要语言
Python
星标
502
派生
48
PR 合并指标
30 天内没有已合并 PR

描述

### Issue: Add `.evaluate` and `.generate_eval_dataset` Methods for RAG Pipelines

**Description:**
Enhance the RAG framework by implementing `.evaluate` and `.generate_eval_dataset` methods to enable easy and automated evaluation of different RAG pipelines.

**Requirements:**
1. **`.evaluate` Method:**
- Allow users to benchmark retrieval performance using standard metrics (e.g., Precision, Recall, F1).
- Support configuration for top-k retrieval and similarity scoring.

2. **`.generate_eval_dataset` Method:**
- Automatically create datasets for evaluation with annotated ground truth queries and responses.
- Include support for custom datasets.

3. **Integration:**
- Seamlessly integrate with existing RAG pipelines.

**Impact:**
Simplifies evaluation workflows and helps compare RAG pipeline performance effectively.

**Priority:** Medium

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。