giuseppec / giuseppec/iml

Slow calculation of Partial Dependence Data and Plot

Open
#222 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
503
Forks
87
PR merge metrics
No merged PRs in 30d

Description

Dear @giuseppec

we are trying out the iml package in combination with mlr3. We experience quite slow calculations.

Here is some raw code. Maybe I can post some reproducible code later.

Our dataset has around 1 million observations and around 50 variables.

```
# LightGBM Learner
learner = mlr_learners$get("classif.lightgbm")
split = partition(task, ratio = 0.67)
learner$train(task, row_ids = split$train)

task_x = task$data(rows = split$test, cols = task$feature_names)
task_y = task$data(rows = split$test, cols = task$target_names)

predictor = Predictor$new(learner, data = task_x, y = task_y)

effect <- FeatureEffect$new(predictor, feature = "my_variable", method = "pdp+ice", grid.size = 3)
effect$plot(rug = FALSE)
```
The slow functions are `FeatureEffect$new` and `effect$plot`.

I programmed it myself with data.table and was able to make it around 10 times faster. In my code I just replaced the variable in data.table and predicted on this new data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.