Evaluation measures not evaluated?
Open
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 755
- Forks
- 128
- PR merge metrics
- No merged PRs in 30d
Description
I was looking at evaluation measures on the australia dataset:
https://www.openml.org/t/125923
It looks like "predictive accuracy" shows many more runs than "area under roc curve".
Why is that?
I also tried "mean recall" (aka balanced accuracy), but that showed no runs at all.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Australia dataset evaluation page at https://www.openml.org/t/125923 and compare the runs reported for predictive accuracy, area under ROC curve, and mean recall. Trace how these evaluation measures are recorded or displayed, then document or correct the reason the latter measures have fewer or no runs.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100