gomate-community / gomate-community/rageval

Data quality metrics

Open
#71 0 comments 0 reactions 9 assignees Claimed by @youngbeauty250 View on GitHub
Dominant language
Python
Stars
172
Forks
10
PR merge metrics
No merged PRs in 30d

Description

In this issue, we discuss the potential metric used to evaluate the quality of an input dataset. The quality of dataset is very important since there are many automatically generated dataset built on ChatGPT.

Some potential quality dimensions related to a QA dataset may be:
1) the number of instances;
2) the diversity of the question topic;
3) the length of the answer;
4) the fluency of the answer;
5) the correctness of the answer;
6) the length of contexts;
7) the relationship between the context and the answer to a question.

we can dicuss it in this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.