posterior_* functions ignore structure of model

Summary:

I am getting a structurally wrongly simulated posterior whenever I have multiple new group levels for a hierarchical model which I want to sample for.... at least for my expectations.

Description:

More details are in the respective brms issue: https://github.com/paul-buerkner/brms/issues/956

Reproducible Steps:

library(rstanarm)
library(mvtnorm)
library(dplyr)

set.seed(425435)

R <- matrix(c(1, 0.1, 0.1, 1), 2, 2, byrow=TRUE)
R

J <- 20
group_draw <- matrix(rmvnorm(J, sigma=R), ncol=2, dimnames=list(NULL, list("a", "b"))) %>%
    as.data.frame() %>%
    mutate(id=1:J)

group_draw

fake  <- expand.grid(id=1:J, x=seq(0,1,length=11)) %>%
    left_join(group_draw) %>%
    arrange(id, x) %>%
    mutate(y_hat= a + b * x, y = rnorm(n(), y_hat), id=factor(id))

head(fake)

fake

fit  <- stan_glmer(y ~ 1 + x + (1+x|id), data=fake, family=gaussian, cores=4)


fit

df  <- data.frame(id=factor((J+1) : (2*J)), x=c(0))

df

plin1  <- posterior_linpred(fit, newdata=df, allow_new_levels=TRUE)
plin2  <- posterior_linpred(fit, newdata=df, allow_new_levels=TRUE)


plin1[1,]
plin2[1,]

The output is from the last two lines:

        1         2         3         4         5         6         7         8
-1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091
        9        10        11        12        13        14        15        16
-1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091
       17        18        19        20
-1.278091 -1.278091 -1.278091 -1.278091
        1         2         3         4         5         6         7         8
-1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091
        9        10        11        12        13        14        15        16
-1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091 -1.278091
       17        18        19        20
-1.278091 -1.278091 -1.278091 -1.278091

Thus, the sampled new level of the random effect is the same for all new subjects, since we have many new subjects here and not just one. So instead of sampling an entire new set of subjects, we just sample one which ignores the structure of the model.

The fix implemented in brms is the right thing from my perspective. In case this is a supported feature, then I would appreciate improved documentation.

RStanARM Version:

2.21.1

R Version:

4.0.2

Operating System:

OS X 10.15.6

Start by running the reproducible R example in the issue and compare repeated posterior_linpred calls with multiple new id levels. Trace the posterior_* functions involved in handling allow_new_levels=TRUE and compare their behavior with the linked brms issue. Done means new group levels follow the model structure, or the supported behavior is clearly documented.