stan-dev / stan-dev/projpred

Projpred::cv_varsel() returning “Not enough (non-NA) data to do anything meaningful” error

Open
#239 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

additive
Dominant language
R
Stars
114
Forks
31
PR merge metrics
No merged PRs in 30d

Description

Hello,

When attempting to perform projection predictive variable selection (projpred::cv_varsel()) on a model that includes both an s() spline term(s) and a random effect(s) I get the following message:
variable_selection <- cv_varsel(ref_model)
[1] “Computing LOOs…”
|| 0%
Error in model.matrix.gamm4(delete.response(terms(formula)), random = random, :
Not enough (non-NA) data to do anything meaningful

I have tried installing the latest development version of the projpred package from GitHub to resolve the issue, but the problem persists. I have also tried to rerun the model and variable selection on a Linux-based server with R (4.1.0) and RStudio Server to test whether the problem is not specific to my local Mac/system, but the process stopped with the same error.

I am attaching two small datasets (one with a Gaussian and the other with a binary response) and two reproducible examples that give the error.

#gaussian_data <- read.csv("~/ ..... /gaussian_data.csv") # define pathway

fit1 <- brm(y ~ s(x) + (1|group)
,data = gaussian_data
,cores = 4
,backend = "cmdstanr"
,control = list(
adapt_delta = 0.99
,max_treedepth = 15)
,seed = 123)
ref_fit1 <- get_refmodel(fit1)
variable_selection_fit1 <- cv_varsel(ref_fit1)

#binary_data <- read.csv("~/ ..... /binary_data.csv") # define pathway

fit2 <- brm(y ~ s(x) + (1|group)
,data = binary_data
,family = bernoulli(link = "logit")
,cores = 4
,backend = "cmdstanr"
,control = list(
adapt_delta = 0.99
,max_treedepth = 15)
,seed = 123)
ref_fit2 <- get_refmodel(fit2)
variable_selection_fit2 <- cv_varsel(ref_fit2)

Many thanks for looking at this.

Best wishes,

Tom

gaussian_data.csv
binary_data.csv

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the two inline brms examples with gaussian_data.csv and binary_data.csv, starting at get_refmodel() and cv_varsel() where “Computing LOOs…” begins. Compare the spline-plus-random-effect cases and confirm that variable selection completes without the non-NA data error for both responses.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.