Draw order inconsistent with `posterior_linpred` and `posterior_epred`
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 401
- Forks
- 136
- PR merge metrics
- No merged PRs in 30d
Description
Summary:
It appears that every invocation of posterior_linpred and posterior_epred gives a different permutation of the draws. This prevents users from doing posterior calculations (e.g. covariances) using the draws.
Description:
It can be useful to use the output of posterior_linpred and posterior_epred in further computation, e.g., as part of covariances with model parameters. As far as I can tell this is currently impossible, since each invocation seems to return a different random permutation of the draws.
Reproducible Steps:
# Simulate some logistic regression data
set.seed(42)
n_obs <- 1000
num_cat <- 3
rel_min <- 0.1
cat_probs <- runif(num_cat)
cat_probs[cat_probs < rel_min] <- rel_min
cat_probs <- cat_probs / sum(cat_probs)
x <- sample(num_cat, size=n_obs, replace=TRUE, prob=cat_probs)
y_prob <- runif(num_cat)
df <-
data.frame(x=x, p=y_prob[x]) %>%
mutate(y=as.numeric(runif(n()) < p))
num_draws <- 5000
# Run rstanarm
fit <- stan_glmer(y ~ (1 | x),
family = binomial(link = "logit"),
data = df,
prior = normal(0, 1, autoscale = TRUE),
prior_covariance = decov(scale = 0.50),
adapt_delta = 0.99,
refresh = 0,
seed = 1010,
iter = num_draws)
# Then every one of these repeated commands gives a different answer each time:
posterior_linpred(fit, newdata=df, draws=num_draws, seed=42, permuted=FALSE)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws, seed=42, permuted=FALSE)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws, seed=42)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws, seed=42)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws - 10)[1,1]
posterior_linpred(fit, newdata=df, draws=num_draws - 10)[1,1]
RStanARM Version:
rstanarm Version: 2.21.4
R Version:
4.1.2
Operating System:
Operating System: Ubuntu 22.04.4 LTS
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the behavior through the posterior_linpred and posterior_epred entry points using the R example in the issue. Trace how draws and seeds are selected across repeated calls, then verify that related posterior calculations preserve draw correspondence and that the provided reproducible commands no longer return inconsistent permutations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100