huggingface / huggingface/evaluate

matthews correlation for multiple features?

Open
#317 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

I'd like to add matthews correlation for the multilabel case.
This essentially has a few options:

* ("micro") flatten predictions and targets, then calculate
* ("macro") calculate it per-feature and average
* Implement micro, macro as in f1 score.

Since scikit-learn doesn't support this out of the box, this may also be a terrible idea for some reason, in which case I'd like to learn why.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.