huggingface / huggingface/evaluate
Evaluator should handle metric kwargs
Open
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 341
- PR merge metrics
- No merged PRs in 30d
Description
Currently the evaluator's `prepare_metric` function does not take metric kwargs; they should be passed all the way down and the call in `prepare_metric` should be changed to `evaluate.load(module_name, **kwargs)`
cc @NimaBoscarino so we don't forget :)
Contributor guide
Assessment
This issue has not been assessed yet.