stan-dev / stan-dev/rstanarm

Noting which tests fail in feature/survival branch (mostly frailtymod)

Open
#584 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
401
Forks
136
PR merge metrics
No merged PRs in 30d

Description

Summary:

There are a dozen of failing tests in the feature/survival branch, specific to the test_stan_surv.R function. All appear to be related to the frailty model.

These are run against a WIP branch at https://github.com/jburos/rstanarm/tree/review-failing-tests, which is a fork of feature/survival branch in this repo that:

  1. removes a few of the JM or mvmer-specific changes in the feature/survival branch, and
  2. updates the syntax for tests to be consistent with current conventions.

Note: Some of these may be addressed by the sparse matrix fixes -- analogous to https://github.com/stan-dev/rstanarm/commit/22d60534e52ab50f7e521d8a77aae0d6aa8114c2 but for the survival model -- which I have in a separate branch. Reporting this here for the sake of transparency.

Description:

These are the failures:

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.521 - -2.3 == 2.82

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.487 - 0.3 == 0.187

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] -1.01 - -2.3 == 1.29

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.765 - 1 == -0.235

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.486 - 0.3 == 0.186

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.698 - -2.3 == 3

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 0.797 - 1 == -0.203

── Failure (test_stan_surv.R:776:3): (code run outside of `test_that()`) ───────
get_ests(modfrail)[[i]] not equal to true[[i]].
1/1 mismatches
[1] 1.25 - -2.3 == 3.55

[ FAIL 12 | WARN 649 | SKIP 0 | PASS 381 ]
Reproducible Steps:

After cloning the fork mentioned above:

git clone https://github.com/jburos/rstanarm/tree/review-failing-tests
cd rstanarm
git checkout review-failing-tests

Run the survival-model related tests.

> devtools::load_all()
> testthat::test_file('tests/testthat/test_stan_surv.R')
RStanARM Version:
> packageVersion("rstanarm")
[1] ‘2.21.2’
R Version:
> getRversion()
[1] ‘4.2.0’
Operating System:
─ Session info ───────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.2.0 (2022-04-22)
 os       macOS 13.1
 system   x86_64, darwin17.0
 ui       RStudio
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       America/New_York
 date     2023-02-07
 rstudio  2022.12.0+353 Elsbeth Geranium (desktop)
 pandoc   2.19.2 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the feature/survival branch and run tests/testthat/test_stan_surv.R after loading the package with devtools::load_all(). Inspect the failures around line 776 and the modfrail estimates, comparing them with the sparse-matrix changes noted in the issue. Done means the twelve frailty-related failures are explained and the survival test file passes or has justified expectations.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.