macro weighted recall is the same as accuracy
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 755
- Forks
- 128
- PR merge metrics
- No merged PRs in 30d
Description
At this moment OpenML calculates the recall as macro weighted recall. But if you weight the TPR of each class by class size, isn't that exactly the same as accuracy? I don't know whether OpenML allows for information retrieval-like tasks where you can set one class as "relevant" and the other as "non-relevant". But if it doesn't, I am unsure what the added value is of the recall measure. Unless the unweighted versions are added as well, as suggested in #627.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating where OpenML calculates recall and compare the current macro weighted recall definition with accuracy for multiclass tasks. Read the discussion in #627 to understand the proposed unweighted metrics, then clarify whether the desired outcome is a correction or additional measures and define tests for the resulting metric values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100