stan-dev / stan-dev/rstan

Threshold for R-hat (1.01 or 1.05)

Open
#812 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
R
Stars
1.1k
Forks
266
Avg merge
2h 56m
Merged PRs (30d)
1

Description

Summary:

In rstan:::throw_sampler_warnings() and on the help page for rstan::Rhat(), the threshold of 1.05 is used/recommended for checking the "new" R-hat, whereas 1.01 is the threshold recommended at other (more recent?) places. I am uncertain whether rstan's current behavior is intended or a bug.

Description:

It seems like 1.01 is the currently recommended threshold for the "new" R-hat (as proposed by Vehtari et al. (2020) and as implemented in rstan::Rhat()), see Vehtari et al. (2020) and the Brief Guide to Stan's Warnings. However, in rstan:::throw_sampler_warnings() and on the help page for rstan::Rhat(), the (older) threshold of 1.05 is used or recommended, respectively. Is this intended or a bug?

Btw, the reference for Vehtari et al. (2020) needs to be updated at some places:

  • the help page for rstan::Rhat()
  • the "Brief Guide to Stan's Warnings"
  • ... (perhaps other places)
Reproducible Steps:

May be seen from the source code of rstan:::throw_sampler_warnings() and ?rstan::Rhat.

Current Output:

Not applicable.

Expected Output:

Not applicable.

RStan Version:

I have 2.19.3 installed, but this also seems to apply to 2.21.1.

R Version:

This is independent of the R version.

Operating System:

This is independent of the OS.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare the threshold in rstan:::throw_sampler_warnings() with the implementation and help page for rstan::Rhat(), then review the linked Brief Guide to Stan's Warnings and Vehtari et al. (2020). Determine the intended recommendation and update the inconsistent warning behavior, documentation, and references so they agree.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.