Issues with check_model() for Stan models

I'm not exactly what the thinking is behind the current behavior of check_model() for Stan models. It calls the internal function performance:::.check_assumptions_stan(). This returns a data frame with columns Group (Prior or Posterior), y (parameter), x (value), and id (posterior draw).

I'm not exactly sure what sort of plot this is intended to be used for (perhaps a scatterplot matrix for the various parameters across posterior draws?). Could someone elaborate? @strengejacke @mattansb? Nevertheless, it is really different from the results for lm/glm models, where a list with the various diagnostic data frames is returned. This produces several problems:

1. see::plot.see_check_model() fails for check_model(stan_model) objects

   • This function expects to find the names of plots in a list (QQ, VIF, etc.). It errors when given the single data frame from performance:::.check_assumptions_stan().

2. There is no way for users to generate the typical regression model diagnositic plots, such as fitted-residual plots, qq plots, etc.

3. The current performance:::.check_assumptions_stan() function fails often for brmsfit models

   • If sample_priors = "no" (default), it stops. It could instead return a data frame with just the posterior.
   • If improper priors are used (default), the reshaping fails. It should be robust to the non-sampled priors.

This leads to 2 big questions:

1. What should check_model() do for Stan models?

   • My preference would be for it to return the same sort of plots as for lm/glm objects.
   • Could you describe what sort of plot was intended with the current output? Could we implement that as a different function (similar to pp_check())?

2. What should we do in the mean time?

   • My suggestion would be to internally call bayestestR::convert_bayesian_as_frequentist(), then call check_model() on that. I think this would be an okay stopgap until we implement the individual plots for stan objects.

Thoughts?

Start by reading check_model() and performance:::.check_assumptions_stan(), then inspect see::plot.see_check_model() to understand the incompatible data shape. Review the described lm/glm behavior and brmsfit failure cases before proposing the intended Stan diagnostics. Done requires an agreed design and compatible behavior for Stan models, including the stated prior-sampling cases.