huggingface / huggingface/evaluate

how to see the metrics in an EvaluateModule & comparison of EvlauationModules

Open
#334 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

After loading metrics from a configuration, say metrics = evaluate.load("glue","mnli"), how can I access what individual metrics the EvaluateModule would compute (I couldn't find an easy way to do this from the documenation). Also is there a way to check if two EvaluateModule are of the same type -- i.e., do they calculate the same metrics and take same inputs?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.