"thin" argument affects diagnostics
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 238
- Forks
- 106
- Avg merge
- 2h 29m
- Merged PRs (30d)
- 6
Description
Describe the bug
If I use the same seed (to ensure reproducibility) and do two model$fit() runs with identical arguments except that one has thin=4, I get fewer divergent transitions, fewer max_treedepth transitions, and worse EBFMI reported for the run that has thinning. (Probably an underlying Stan issue, not just cmdstanr)
To Reproduce
model <- cmdstanr::cmdstan_model(path_to_funnel_model)
fit1 <- model$sample(data=list(), seed=12345678, chains=1, iter_warmup=1000, iter_sampling=4000)
fit1$diagnostic_summary()
# ^^^ num_divergent = 4, num_max_treedepth = 2, ebfmi = 0.08203262
fit2 <- model$sample(data=list(), seed=12345678, chains=1, iter_warmup=1000, iter_sampling=4000, thin=4)
fit2$diagnostic_summary()
# ^^^ num_divergent = 0, num_max_treedepth = 1, ebfmi = 0.2993997
Expected behavior
Thinning should only affect the number of draws saved, not the diagnostics. A divergent transition is an indication of a possible problem whether or not you save it.
Operating system
Your operating system (e.g. mac os x 10.15, windows 10, etc.)
CmdStanR version number
Your CmdStanR version number (e.g. from packageVersion("cmdstanr")).
Additional context
cmdstanr version 0.6.0.9000
R version 4.2.2
macOS 13.4 (Ventura)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the two model$sample reproductions with identical seeds and compare their diagnostic_summary() results. Trace how thin is handled between sampling and diagnostics, then verify that divergent transitions, max_treedepth transitions, and EBFMI are unchanged except for the saved draw count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, r
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100