stan-dev / stan-dev/rstanarm

Problem with using `kfold` with stansurv objects?

Open
#473 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

survival
Dominant language
R
Stars
401
Forks
136
PR merge metrics
No merged PRs in 30d

Description

Summary:

There appears to be a problem with using kfold on stan_surv models. Error returned is: Error: passing unknown arguments: subset.

Description:

I'm fitting multi-level survival models with time-varying using rstanarm::stan_surv models and want to use kfold cross validation where groups of individuals are heldout (individuals are repeatedly surveyed overtime). I tried aggregating the log likelihoods as done in the time-varying coefficient example in the Arxiv paper (https://arxiv.org/pdf/2002.09633.pdf), however I get pretty poor Pareto k diagnostic values. So I thought using kfold might be a better alternative, where I group individuals into folds. However, when I do this I get the following error Error: passing unknown arguments: subset.
NOTE I also get this error if I don't stratify/group the data into folds, and if I exclude the random effect for individual.
Any advice welcomed!

Reproducible Steps:

Taking the reproducible example presented in the Arxiv paper:

library(rstanarm)
library(survival)

dat <- survival::pbc
dat <- dat[dat$id <= 312, ]

dat <- tmerge(dat, dat, id = id,death = event(time, as.numeric(status == 2)))

dat <- tmerge(dat, survival::pbcseq, id = id,ascites  = tdc(day, ascites),
              bili = tdc(day, bili),
              albumin = tdc(day, albumin),
              protime = tdc(day, protime),
              alk.phos = tdc(day, alk.phos))

dat <- dat[, c("id", "tstart", "tstop", "death", "bili", "protime")]

mod_tvc <- stan_surv(formula = Surv(tstart, tstop, death) ~ log(bili) + log(protime) + (1|id), data = dat)

inds <- loo::kfold_split_grouped(K= 10, x =mod_tvc$data$id)
kfold(mod_tvc, folds = inds)
Fitting model 1 out of 10
Error: passing unknown arguments: subset.
RStanARM Version:
packageVersion("rstanarm")
[1] ‘2.21.2’

packageVersion("loo")
[1] ‘2.3.1’
R Version:
getRversion()
[1] ‘4.0.1’
Operating System:

OSX 10.14.6

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the supplied stan_surv example with loo::kfold_split_grouped and kfold(mod_tvc, folds = inds), then investigate how kfold handles stan_surv models and the subset argument. Done means the grouped and ungrouped kfold calls no longer fail with “passing unknown arguments: subset.”

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.