huggingface / huggingface/evaluate
Statistical significance testing
Open
metric request
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
As mentioned in a recent [paper](https://aclanthology.org/2021.acl-long.566.pdf) about evaluating MT approaches (and probably other sources too), statistical significance testing can be used to confirm that one method is superior to another, saying that 'it remains one of the most cost-effective tools to check how trustworthy a particular difference between two metric scores is."
We could possibly use the Wilcoxon signed-rank test (implemented in [scipy](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wilcoxon.html)) or another similar approach.
Contributor guide
Assessment
This issue has not been assessed yet.