stan-dev / stan-dev/loo

Error when running loo with moment_match = TRUE on rstan model from previous session

Open
#183 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
157
Forks
38
Avg merge
4d 16h
Merged PRs (30d)
2

Description

Hi,

I'm trying to run loo with moment matching on an rstan model fit in a previous session. The stanfit object was saved using:

fit <- stan(...)
saveRDS(fit,...)

And then loaded and loo (from https://github.com/stan-dev/rstan/blob/develop/rstan/rstan/R/loo.R) was run with:

fit <- loadRDS(...)
loo(fit, moment_match = TRUE)

which throws the following error:

Error in .local(object, ...) : 
  the model object is not created or not valid
Calls: loo ... apply -> FUN -> <Anonymous> -> <Anonymous> -> .local
Execution halted

Fitting and loading are both happening on the same system (university HPC cluster). I have no problems loading the stanfit object and then running other functions on this system (running extract(fit) for example) or loading the stanfit object on my local system (Mac OS 11.2.1). This also works fine if the model is fit and then loo is run within the same session (i.e. not from a .rds file).

HPC Cluster specs:

cat /etc/*-release

Cluster Manager v9.1
slave
NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.2"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.2"
Red Hat Enterprise Linux release 8.2 (Ootpa)
Red Hat Enterprise Linux release 8.2 (Ootpa)

R environment specs:

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
rstan (version 2.21.2)

Thanks for your help and developing such an awesome package! If this is an rstan and not a loo problem I'm happy to move the issue there.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the loo(fit, moment_match = TRUE) entry point and the rstan/R/loo.R implementation linked in the report. Reproduce the failure by saving and reloading a stanfit before calling loo, then compare it with a same-session call. Done means the reloaded model completes moment matching without the invalid-model error, with a regression test covering the saved-and-loaded case.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.