giuseppec / giuseppec/iml

Metrics::f1

Open
#166 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
503
Forks
87
PR merge metrics
No merged PRs in 30d

Description

Hello,

Thanks for the package.

I know that the `Metrics` package is not implemented by you but only exported by your package. But I encountered an issue trying to run `FeatureImp` using the `f1` metric.

![Image Pasted at 2021-2-27 17-48](https://user-images.githubusercontent.com/26466764/109432344-fa262e80-7a0a-11eb-9efc-0c570e059846.png)

The `f1` metric from the `Metrics` package seems to be wrongly implemented. The first two lines are already an indication (`unique`), making the rest of the code also wrong.
```r
library("iml")
Metrics::f1
# function (actual, predicted)
# {
# act <- unique(actual)
# pred <- unique(predicted)
# tp <- length(intersect(act, pred))
# fp <- length(setdiff(pred, act))
# fn <- length(setdiff(act, pred))
# if (tp == 0) {
# return(0)
# }
# else {
# precision <- tp/(tp + fp)
# recall <- tp/(tp + fn)
# return(2 * precision * recall/(precision + recall))
# }
# }
#
#
```

Reproducible example:
```r
actual <- c(0,1,1,0,1,1)
# [1] 0 1 1 0 1 1
predicted <- c(0,1,1,0,0,1)
# [1] 0 1 1 0 0 1
Metrics::f1(actual, predicted)
# [1] 1
```
This is wrong as one would expect (for binary classification) with `1` as the true label, an F1 score of `0.86`. If the positive label were `0` the F1 score would be `0.8`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.