stan-dev / stan-dev/rstanarm

survival: different results with R `v3.6.3` vs `v4.2.0`, but same `rstanarm` version and seed

Open
#582 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

survival
Dominant language
R
Stars
401
Forks
136
PR merge metrics
No merged PRs in 30d

Description

Summary:

Same rstanarm version from feature/survival (version 2.21.2), same seed, different results between R 3.6.3 and 4.2.0.

Description:

As per the summary, I have the same version of rstanarm from feature/survival installed on R version 3.6.3 and 4.2.0. Even with the same seed, I get different draws form the posterior distribution.

The 2 versions of rstanarm were not installed at the same time. Could it be that some changes to the code potentially related to this issue were introduced, yet the version of rstanarm was not updated? Otherwise, do you have any idea about what the different results could be due to?

But above all: do you know I can obtain consistent results between R 3.6.3 and 4.2.0?

Output:

(reproducible code below)

R 3.6.3 + rstanarm 2.21.2

stan_surv
 baseline hazard: weibull
 formula:         Surv(futime, death) ~ mspike
 observations:    241
 events:          225 (93.4%)
 right censored:  16 (6.6%)
 delayed entry:   no
------
              Median    MAD_SD    exp(Median)
(Intercept)   -10.15258   0.62484        NA  
mspike         -0.12206   0.17364   0.88510  
weibull-shape   1.18821   0.06543        NA  

R 4.2.0 + rstanarm 2.21.2

stan_surv
 baseline hazard: weibull
 formula:         Surv(futime, death) ~ mspike
 observations:    241
 events:          225 (93.4%)
 right censored:  16 (6.6%)
 delayed entry:   no
------
              Median    MAD_SD    exp(Median)
(Intercept)   -10.14793   0.74071        NA  
mspike         -0.11830   0.18188   0.88843  
weibull-shape   1.19225   0.07253        NA 
Reproducible Code:
library(survival)
library(rstanarm)

surv <- stan_surv(Surv(futime, death) ~ mspike, 
                  data = mgus, 
                  basehaz = "weibull",
                  chains = 1, 
                  seed = 1234)

print(surv, digits = 5)
RStanARM Version:

2.21.2

R Version:

3.6.3 and 4.2.0

Operating System:

Mac OS 10.14.6

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the reproducible stan_surv example with R 3.6.3 and R 4.2.0, using the stated rstanarm 2.21.2 version, seed, data, and settings. Compare the environments and posterior output to identify the source of the discrepancy; done means explaining the cause and documenting how to obtain consistent results.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.