tidymodels / tidymodels/multilevelmod

Feature Request: Additional documentation / examples on using `recipes`

Open
#75 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation tidy-dev-day :nerd_face:
Dominant language
R
Stars
74
Forks
6
Avg merge
1h 53m
Merged PRs (30d)
1

Description

I'm not sure if I am doing something wrong but when I attempt to follow the example on the page I get an error when using recipes, I suspect that this has to do with the functions add_variables passing columns as-is to the model fitting function, however, if we use step_dummy or step_pca the columns will change.

library('lme4')
#> Loading required package: Matrix
library('multilevelmod')
#> Loading required package: parsnip
library('workflowsets')
library('tidymodels')
tidymodels_prefer()

riesby
#> # A tibble: 250 × 7
#>    subject depr_score  week  male endogenous imipramine desipramine
#>    <fct>        <dbl> <dbl> <dbl>      <dbl>      <dbl>       <dbl>
#>  1 101             -8     0     0          0       4.04        4.20
#>  2 101            -19     1     0          0       3.93        4.81
#>  3 101            -22     2     0          0       4.33        4.96
#>  4 101            -23     3     0          0       4.37        4.96
#>  5 103            -18     0     1          0       2.77        5.24
#>  6 103             -9     1     1          0       3.47        5.21
#>  7 103            -18     2     1          0       3.53        5.34
#>  8 103            -20     3     1          0       3.58        5.36
#>  9 104            -11     0     1          1       5.34        4.75
#> 10 104            -16     1     1          1       5.75        5.06
#> # ℹ 240 more rows

splits <- group_initial_split(riesby, group = subject)

folds <- group_vfold_cv(training(splits), v = 5, group = subject)

rec <- recipe(depr_score ~ ., data = training(splits) %>% filter(week == 0)) %>%
  add_role(subject, new_role = "exp_unit") %>%
  update_role(week, new_role = "id variable") %>%
  step_normalize(all_numeric_predictors(), -all_outcomes(), -has_role("exp_unit") ) %>%
  step_pca(all_numeric_predictors(), -all_outcomes(), -has_role("exp_unit"), num_comp = tune())

lmer_spec <- 
  linear_reg() %>% 
  set_engine("lmer")

lmer_wflow <- 
  workflow() %>% 
  add_variables(outcomes = depr_score, predictors = c(male, endogenous, imipramine, desipramine, subject)) %>% 
  add_model(lmer_spec, formula = depr_score ~ male + endogenous + imipramine + desipramine + (1|subject))



grid_results <- tune_grid(
  lmer_wflow,
  resamples = folds,
  grid = 10,
  metrics = metric_set(rmse),
  control = control_grid(save_pred = T, save_workflow = TRUE)
)
#> Warning: No tuning parameters have been detected, performance will be evaluated
#> using the resamples with no tuning. Did you want to [tune()] parameters?

collect_predictions(grid_results)
#> # A tibble: 187 × 5
#>     .pred id         .row depr_score .config             
#>     <dbl> <chr>     <int>      <dbl> <chr>               
#>  1  -6.14 Resample1     5         -6 Preprocessor1_Model1
#>  2  -6.69 Resample1     6         -6 Preprocessor1_Model1
#>  3  -5.40 Resample1     7         -9 Preprocessor1_Model1
#>  4  -7.04 Resample1     8        -13 Preprocessor1_Model1
#>  5  -7.36 Resample1    23         -6 Preprocessor1_Model1
#>  6  -8.39 Resample1    24         -7 Preprocessor1_Model1
#>  7  -9.80 Resample1    25        -12 Preprocessor1_Model1
#>  8 -10.3  Resample1    26        -13 Preprocessor1_Model1
#>  9  -6.23 Resample1    27         -8 Preprocessor1_Model1
#> 10  -7.15 Resample1    28         -8 Preprocessor1_Model1
#> # ℹ 177 more rows

collect_metrics(grid_results)
#> # A tibble: 1 × 6
#>   .metric .estimator  mean     n std_err .config             
#>   <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
#> 1 rmse    standard    7.35     5   0.687 Preprocessor1_Model1

final_model <- fit_best(grid_results)

final_model
#> ══ Workflow [trained] ══════════════════════════════════════════════════════════
#> Preprocessor: Variables
#> Model: linear_reg()
#> 
#> ── Preprocessor ────────────────────────────────────────────────────────────────
#> Outcomes: depr_score
#> Predictors: c(male, endogenous, imipramine, desipramine, subject)
#> 
#> ── Model ───────────────────────────────────────────────────────────────────────
#> Linear mixed model fit by REML ['lmerMod']
#> Formula: depr_score ~ male + endogenous + imipramine + desipramine + (1 |  
#>     subject)
#>    Data: data
#> REML criterion at convergence: 1177.133
#> Random effects:
#>  Groups   Name        Std.Dev.
#>  subject  (Intercept) 5.577   
#>  Residual             4.585   
#> Number of obs: 187, groups:  subject, 50
#> Fixed Effects:
#> (Intercept)         male   endogenous   imipramine  desipramine  
#>     13.7276       0.6072       1.6161      -0.8234      -4.2021

Now try with recipe

lmer_wflow_rec <- lmer_wflow %>%
  remove_variables() %>%
  add_recipe(rec)

grid_results_rec <- tune_grid(
  lmer_wflow_rec,
  resamples = folds,
  grid = 10,
  metrics = metric_set(rmse),
  control = control_grid(save_pred = T, save_workflow = TRUE)
)
#> → A | error:   object 'male' not found
#> There were issues with some computations   A: x1There were issues with some computations   A: x2There were issues with some computations   A: x5There were issues with some computations   A: x9There were issues with some computations   A: x11There were issues with some computations   A: x14There were issues with some computations   A: x17There were issues with some computations   A: x20There were issues with some computations   A: x20
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied example and inspect how add_variables(), remove_variables(), add_recipe(), step_normalize(), and step_pca() interact during tune_grid(). Use show_notes(.Last.tune.result) to capture the failure details. Done means the documentation or examples clearly explain supported recipe usage and the expected behavior when transformed columns are passed to the mixed-model workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.