ModelOriented / ModelOriented/DALEX
Option to specify random effects?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm wondering if it would be possible (or even make sense) to have the option to specify random effects in the model explainer?
I thought about this because when looking at feature importance, the full model RMSE is quite different to one that accounts for random effects. For example...
library(tidyverse)
library(tidymodels)
library(lme4)
library(DALEXtra)
df <- nlme::Oxboys
df
# model using lmer
lmr_mod <- lme4::lmer(height ~ age + Occasion + (1|Subject), df)
sjstats::rmse(lmr_mod)
# RMSE is 1.2
# model with tidymodels
mixed_model_spec <- linear_reg() %>% set_engine("lmer")
mixed_model_wf <- workflow() %>%
add_model(mixed_model_spec, formula = height ~ age + Occasion + (1|Subject)) %>%
add_variables(outcomes = height, predictors = c(age, Occasion, Subject))
fit <- fit(mixed_model_wf, df)
explainer <-
explain_tidymodels(
fit,
data = dplyr::select(df, c(age, Occasion, Subject)),
y = df$height,
label = "lmm",
verbose = T)
var_imp <-
feature_importance(explainer)
# full model RMSE is 8.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the issue's lme4::lmer, explain_tidymodels(), and feature_importance() example, then trace how the explainer obtains predictions and RMSE. Determine the intended behavior for random effects and how completion would be verified against the reported discrepancy; no specific source file or test is named.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100