stan-dev / stan-dev/projpred

length of penalty for factors

Open
#66 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
R
Stars
114
Forks
31
PR merge metrics
No merged PRs in 30d

Description

Hi,
when there is a factor as a predictor in a brms model and when a penalty vector is supplied to cv_varsel(), the variable selection seems to have issues with the penalty vector; this behavior started after the latest brms update (see session info below).

    # from the tutorial
    library(projpred)
    library(brms)
    data('df_gaussian', package = 'projpred')
    split_structure <- break_up_matrix_term(y ~ x, data = df_gaussian)
    df_gaussian <- split_structure$data
    formula <- split_structure$formula
    d <- df_gaussian
    n <- nrow(df_gaussian) # 100
    D <- ncol(df_gaussian[, -1]) # 20
    p0 <- 5 # prior guess for the number of relevant variables
    tau0 <- p0/(D-p0) * 1/sqrt(n) # scale for tau (notice that stan_glm will automatically scale this by sigma)

    # Make a factor out of one variable <--------------------
    df_gaussian$x20 <- as.factor(sample(0:1, nrow(df_gaussian), rep=T))

    fit <- brm(formula, family=gaussian(), data=df_gaussian,
               prior=prior(horseshoe(scale_global = tau0, scale_slab = 1), class=b),
               seed=1, chains=2, iter=500)
    # make penalty
    betas <- grep("^b", parnames(fit), value=TRUE)[-1]
    penalty <- sample(0:1, length(betas), rep = T)
  
    cvs <- cv_varsel(fit, method = "L1", penalty = penalty)
    # >    
    # [1] "Computing LOOs..."  
    # Error in glm_elnet(d_train$x, mu, family, lambda_min_ratio = opt$lambda_min_ratio,  :      
    # Incorrect length of penalty vector (should be 21). 

    # making the penalty one longer
    cvs <- cv_varsel(fit, method = "L1", penalty = c(penalty,1))
    # >
    # warning: solve(): system seems singular; attempting approx solution

Session Info

R version 4.0.2 (2020-06-22)                                                             
Platform: x86_64-w64-mingw32/x64 (64-bit)                                                
Running under: Windows 10 x64 (build 18363)                                              
                                                                                         
Matrix products: default                                                                 
                                                                                         
locale:                                                                                  
[1] LC_COLLATE=German_Switzerland.1252  LC_CTYPE=German_Switzerland.1252                 
[3] LC_MONETARY=German_Switzerland.1252 LC_NUMERIC=C                                     
[5] LC_TIME=German_Switzerland.1252                                                      
                                                                                         
attached base packages:                                                                  
[1] stats     graphics  grDevices utils     datasets  methods   base                     
                                                                                         
other attached packages:                                                                 
[1] brms_2.14.4    Rcpp_1.0.5     projpred_2.0.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied tutorial reproduction using cv_varsel() with method="L1" and a brms model containing a factor predictor. Trace how the penalty vector is sized before the glm_elnet() call, then verify that the original penalty length is accepted without the reported length error or singular-system warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.