Error in `[.data.frame`(prediction, , self$class, drop = FALSE) : undefined columns selected
- Dominant language
- R
- Stars
- 503
- Forks
- 87
- PR merge metrics
- No merged PRs in 30d
Description
There may be an error in the tutorial of this package on this [website](https://uc-r.github.io/iml-pkg): "Interpreting Machine Learning Models with the iml Package". If we want to check the `FeatureImp` of the models, the following error appears: "Error in `[.data.frame`(prediction, , self$class, drop = FALSE) : undefined columns selected", as mentioned in the title. Does any know why this error occurs? Here is a reproducible example:
``` r
library(rsample) # data splitting
library(ggplot2) # allows extension of visualizations
library(dplyr) # basic data transformation
library(h2o) # machine learning modeling
library(iml) # ML interprtation
h2o.no_progress()
h2o.init()
#> Connection successful!
#>
#> R is connected to the H2O cluster:
#> H2O cluster uptime: 8 hours 33 minutes
#> H2O cluster timezone: Europe/Amsterdam
#> H2O data parsing timezone: UTC
#> H2O cluster version: 3.36.1.2
#> H2O cluster version age: 2 months
#> H2O cluster name: H2O_started_from_R_quinten_xns163
#> H2O cluster total nodes: 1
#> H2O cluster total memory: 3.64 GB
#> H2O cluster total cores: 8
#> H2O cluster allowed cores: 8
#> H2O cluster healthy: TRUE
#> H2O Connection ip: localhost
#> H2O Connection port: 54321
#> H2O Connection proxy: NA
#> H2O Internal Security: FALSE
#> R Version: R version 4.1.0 (2021-05-18)
#data
library(modeldata)
data("attrition", package = "modeldata")
#classification data
df <- attrition %>%
mutate_if(is.ordered, factor, ordered = FALSE) %>%
mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))
# convert to h2o object
df.h2o <- as.h2o(df)
# create train, validation, and test splits
set.seed(123)
splits <- h2o.splitFrame(df.h2o, ratios = c(.7, .15), destination_frames = c("train","valid","test"))
names(splits) <- c("train","valid","test")
# variable names for resonse & features
y <- "Attrition"
x <- setdiff(names(df), y)
# elastic net model
glm <- h2o.glm(
x = x,
y = y,
training_frame = splits$train,
validation_frame = splits$valid,
family = "binomial",
seed = 123
)
# random forest model
rf <- h2o.randomForest(
x = x,
y = y,
training_frame = splits$train,
validation_frame = splits$valid,
ntrees = 1000,
stopping_metric = "AUC",
stopping_rounds = 10,
stopping_tolerance = 0.005,
seed = 123
)
#> Warning in .h2o.processResponseWarnings(res): early stopping is enabled but neither score_tree_interval or score_each_iteration are defined. Early stopping will not be reproducible!.
# gradient boosting machine model
gbm <- h2o.gbm(
x = x,
y = y,
training_frame = splits$train,
validation_frame = splits$valid,
ntrees = 1000,
stopping_metric = "AUC",
stopping_rounds = 10,
stopping_tolerance = 0.005,
seed = 123
)
#> Warning in .h2o.processResponseWarnings(res): early stopping is enabled but neither score_tree_interval or score_each_iteration are defined. Early stopping will not be reproducible!.
# model performance
h2o.auc(glm, valid = TRUE)
#> [1] 0.7870935
h2o.auc(rf, valid = TRUE)
#> [1] 0.7681021
h2o.auc(gbm, valid = TRUE)
#> [1] 0.7468242
# 1. create a data frame with just the features
features <- as.data.frame(splits$valid) %>% select(-Attrition)
# 2. Create a vector with the actual responses
response <- as.numeric(as.vector(splits$valid$Attrition))
# 3. Create custom predict function that returns the predicted values as a
# vector (probability of purchasing in our example)
pred <- function(model, newdata) {
results <- as.data.frame(h2o.predict(model, as.h2o(newdata)))
return(results[[3L]])
}
# example of prediction output
pred(rf, features) %>% head()
#> [1] 0.18181818 0.27272727 0.06060606 0.54545455 0.03030303 0.42424242
# create predictor object to pass to explainer functions
predictor.glm <- Predictor$new(
model = glm,
data = features,
y = response,
predict.function = pred,
class = "classification"
)
predictor.rf <- Predictor$new(
model = rf,
data = features,
y = response,
predict.function = pred,
class = "classification"
)
predictor.gbm <- Predictor$new(
model = gbm,
data = features,
y = response,
predict.function = pred,
class = "classification"
)
#compute feature importance with specified loss metric
imp.glm <- FeatureImp$new(predictor.glm, loss = "mse")
#> Error in `[.data.frame`(prediction, , self$class, drop = FALSE): undefined columns selected
imp.rf <- FeatureImp$new(predictor.rf, loss = "mse")
#> Error in `[.data.frame`(prediction, , self$class, drop = FALSE): undefined columns selected
imp.gbm <- FeatureImp$new(predictor.gbm, loss = "mse")
#> Error in `[.data.frame`(prediction, , self$class, drop = FALSE): undefined columns selected
```
Created on 2022-07-26 by the [reprex package](https://reprex.tidyverse.org) (v2.0.1)
Thank you in advance!
Contributor guide
Assessment
This issue has not been assessed yet.