huggingface / huggingface/evaluate
how to see the metrics in an EvaluateModule & comparison of EvlauationModules
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
After loading metrics from a configuration, say metrics = evaluate.load("glue","mnli"), how can I access what individual metrics the EvaluateModule would compute (I couldn't find an easy way to do this from the documenation). Also is there a way to check if two EvaluateModule are of the same type -- i.e., do they calculate the same metrics and take same inputs?
Contributor guide
Assessment
This issue has not been assessed yet.