ModelOriented / ModelOriented/DALEX

Error when examining aggregated SHAP values in R using tidymodels

Open
#570 6 comments 0 reactions 1 assignee View on GitHub

@maksymiuks is already working on this.

Since Sep 3, 2024.

bug 💣 question ❔ R 🐳
Dominant language
Python
Stars
1.5k
Forks
172
PR merge metrics
No merged PRs in 30d

Description

I have trained and fitted a C5.0 model with adaptive boosting to predict a binary outcome. This was performed using tidymodels and the parsnip package.

I am now trying to evaluate the model in more detail using DALEX and DALEXtra.

The outcome variable is "yes/no" so I have converted these to binary "1/0". I have created an explainer using the following code with the DALEXtra package:

explain_c50 <- 
  explain_tidymodels(model = c5_final_fit, 
                     data = final_data_test, 
                     y = y_test, 
                     verbose = F)

I have also created an explainer using only the DALEX package, and encounter the exact same issue:

custom_predict <- function(object, newdata) {
  pred <- 
    predict(object, newdata, type = 'prob')[1] %>% 
    pull(.pred_F)
  
  return(pred)
}


DALEX_explainerTest <- DALEX::explain(model = c5_final_fit, 
                                  data = final_data_test,
                                  predict_function = custom_predict,
                                  y = y_test, 
                                  label = "c50-train")

These run without error. However, when I try to run a command to interrogate additive SHAP values I get the following error:

DALEX::shap_aggregated(explainer = explain_c50, new_observations = final_data_test[1:10, ])

Error message is always the same, no matter what I try:

Error in `[<-.data.frame`(`*tmp*`, , candidate, value = list(pure_model_prediction = list( : 
  replacement element 1 is a matrix/data frame of 1 row, need 21

I have tried processing the data slightly differently, tried using the training data (rather than test), etc. The error message is always this way so I assume I am making a fairly fundamental error. I cannot understand why I am getting this error.

Does anyone reading this have any idea?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.