huggingface / huggingface/evaluate

Standardize metric ranges

Open
#120 0 comments 0 reactions 4 assignees Claimed by @lhoestq View on GitHub
enhancement
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

Several common metrics, like `exact_match` and `f1`, sometimes range from 0-1, and sometimes from 0-100.

As discussed with @lhoestq , we think it makes more sense to report them from 0-1, which would entail changing the code of metrics such as [CUAD](https://github.com/huggingface/datasets/blob/master/metrics/cuad/cuad.py).

@emibaylor and I will add other metrics here that we see reporting scores from 0-100 instead of 0-1.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.