huggingface / huggingface/evaluate
Standardize metric ranges
Open
enhancement
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Several common metrics, like `exact_match` and `f1`, sometimes range from 0-1, and sometimes from 0-100.
As discussed with @lhoestq , we think it makes more sense to report them from 0-1, which would entail changing the code of metrics such as [CUAD](https://github.com/huggingface/datasets/blob/master/metrics/cuad/cuad.py).
@emibaylor and I will add other metrics here that we see reporting scores from 0-100 instead of 0-1.
Contributor guide
Assessment
This issue has not been assessed yet.