dotnet / dotnet/machinelearning-modelbuilder
Consider showing balanced accuracy instead of accuracy + additional accuracy metrics
- Dominant language
- Dockerfile
- Stars
- 285
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
Currently, Model builder shows accuracy for the user because it is intuitive to understand.
However, in cases on imbalanced datasets, accuracy can be misleading. I believe real-world datasets are rarely perfectly balanced and in this cases it is no longer intuitive to understand what accuracy means.
For examples, I might have 10% false and 90% true in my dataset. If Model would predict everything as true I still get 90% accuracy.
The issues may even be worse in near-balanced datasets, for example, 60/40% because user might not even think about this possibility.
Even if ML.NET does not have average / balanced accuracy score, it can be calculated from other metrics.
Alternatives for, or additions to, balanced accuracy:
- show the distribution of test data across classes.
- show confusion matrix
- show null accuracy
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.