summary function optional argument use_cache should take into account include

Summary:

The summary function's optional argument use_cache stores all summary statistics in the cache. I believe it should include or exclude parameters based on the include optional argument.

Description:

This issue is based on the discussion here
https://groups.google.com/forum/#!category-topic/stan-users/general/TxWN-PDihlc
on the Stan user group (titled "Print Slow, Even with Excluded Variables")

Many rstan functions contain the ability to set a character vector (pars) of parameters to include or exclude based on a binary variable (include). This is helpful in cases where the summary statistics of one or more of the parameters are computationally costly to evaluate.

The documentation for the summary function mentions pars, but it does not mention include. Nevertheless, include works perfectly well with summary.

However, the optional argument use_cache=TRUE will calculate the summary statistics for all variables, rather than just the ones suggested based on pars and include. The results it displays will take into account pars and include. If use_cache=FALSE, then the summary statistics are not saved.

I suggest that use_cache=TRUE work in the following manner:

1. if there are no summary statistics cached, calculate only the ones needed based on pars and include,
2. if some summary statistics needed are cached but not all, only calculate the new ones,
3. if all summary statistics needed are cached, make no new calculation.

As a result of these changes, summary will only calculate the summary statistics needed at the moment they are needed.

This will improve performance of other functions that indirectly call summary, such as print, that can run slowly even when large variables are excluded.

Reproducible Steps:

library(rstan)

model <- "data {
int<lower=0> J; // number of schools
real y[J]; // estimated treatment effects
real<lower=0> sigma[J]; // s.e. of effect estimates
}
parameters {
real mu;
real<lower=0> tau;
real eta[J];
}
transformed parameters {
real theta[J];
for (j in 1:J)
theta[j] = mu + tau * eta[j];
}
model {
target += normal_lpdf(eta | 0, 1);
target += normal_lpdf(y | theta, sigma);
}
"

schools_dat <- list(J = 8,
y = c(28, 8, -3, 7, -1, 1, 18, 12),
sigma = c(15, 10, 16, 11, 9, 11, 10, 18))

fit <- stan(model_code = model, data = schools_dat,
iter = 1000, chains = 4)

#Below calculates the summary statistics for mu, tau, and eta, but does not save them
summary(fit, pars=c("mu", "tau", "eta"), use_cache=FALSE)

#Below is equivalent to what is above, i.e. include is passed to summary correctly
summary(fit, pars=c("theta"), use_cache=FALSE, include=FALSE)

#However, this call stores all variables in the cache
summary(fit, pars=c("theta"), use_cache=TRUE, include=FALSE)

#As a result, this call subsequently takes no time
summary(fit, pars=c("mu", "tau", "eta"), use_cache=TRUE)

RStan Version:

2.12.1

R Version:

R version 3.3.2 (2016-10-31)

Operating System:

Windows 7

Locate the summary function and trace how use_cache, pars, and include are handled. Reproduce the supplied calls to observe which statistics are calculated and cached. Done means only statistics selected by pars and include are calculated or added to the cache, while cached results are reused for later calls.