AnswerDotAI / AnswerDotAI/RAGatouille
Evaluation Module Request : A standard way to evaluate the fine-tuned model
Open
enhancement
good first issue
help wanted
- Dominant language
- Python
- Stars
- 4k
- Forks
- 276
- PR merge metrics
- No merged PRs in 30d
Description
Hey, I just finished my toy project on fine-tuning the ColBertV2 model and I am wondering if there are or will be any standard evaluation datasets and methods wrapped in the package so that we can easily evaluate the fine-tuned model. Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are identified in the issue. Start by reviewing the package's existing fine-tuning and evaluation capabilities, then define which standard datasets and evaluation methods should be supported and what a completed evaluation workflow would produce.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100