Default of `save_iterations` in `optimizing` is `TRUE` while docs says it is `FALSE` (neither one saves the iterations though)

Summary:

It seems that there is an error in the documentation regarding the save_iterations argument as generated quantities are computed at each iteration of the optimizer by default, but not when save_iterations is explicitly set to FALSE. However, no matter what the value of save_iterations is, no iterations are saved.

Reproducible Steps:

library(rstan)
m <- stan_model(
  model_code = '
  parameters {
    real x;
  } 
  model {
    target += (x^2 - 1)^2 + sin(5 * x);
    print("model block");
  }
  generated quantities {
     print("gq block");
  }'
)
f1 <- optimizing(m, init = 0, refresh = 1, iter = 3)
f2 <- optimizing(m, init = 0, refresh = 1, iter = 3, save_iterations = FALSE)
f3 <- optimizing(m, init = 0, refresh = 1, iter = 3, save_iterations = TRUE)

Current Output:

> f1 <- optimizing(m, init = 0, refresh = 1, iter = 3)
Chain 1: model block

Chain 1: model block

Chain 1: Initial log joint probability = 1
Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        1       1.83539      0.272502     0.0248439      0.0545       0.001        5   
Chain 1: model block
model block
model block
model block
model block

Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        2       1.83539    0.00136076     0.0126881           1           1        6   
Chain 1: model block

Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        3        1.8354   0.000307599    0.00420096      0.6687      0.6687        7   
Chain 1: model block

Chain 1: gq block

Chain 1: Optimization terminated normally: 
Chain 1:   Maximum number of iterations hit, may not be at an optima
> f2 <- optimizing(m, init = 0, refresh = 1, iter = 3, save_iterations = FALSE)
Chain 1: model block

Chain 1: model block

Chain 1: Initial log joint probability = 1
Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        1       1.83539      0.272502     0.0248439      0.0545       0.001        5   
Chain 1: model block
model block
model block
model block
model block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        2       1.83539    0.00136076     0.0126881           1           1        6   
Chain 1: model block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        3        1.8354   0.000307599    0.00420096      0.6687      0.6687        7   
Chain 1: model block

Chain 1: gq block

Chain 1: Optimization terminated normally: 
Chain 1:   Maximum number of iterations hit, may not be at an optima
> f3 <- optimizing(m, init = 0, refresh = 1, iter = 3, save_iterations = TRUE)
Chain 1: model block

Chain 1: model block

Chain 1: Initial log joint probability = 1
Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        1       1.83539      0.272502     0.0248439      0.0545       0.001        5   
Chain 1: model block
model block
model block
model block
model block

Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        2       1.83539    0.00136076     0.0126881           1           1        6   
Chain 1: model block

Chain 1: gq block

Chain 1:     Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
Chain 1:        3        1.8354   0.000307599    0.00420096      0.6687      0.6687        7   
Chain 1: model block

Chain 1: gq block

Chain 1: Optimization terminated normally: 
Chain 1:   Maximum number of iterations hit, may not be at an optima

And

identical(f1, f2)
identical(f2, f3)

both return TRUE. The generated files in case of `sample_file = "output.txt" are also identical, having the following contents:

# Point Estimate Generated by Stan
# stan_version_major=2
# stan_version_minor=32
# stan_version_patch=2
# init=0
# enable_random_init=1
# seed=516173459
# chain_id=1
# iter=3
# refresh=1
# save_iterations=1
# algorithm=LBFGS
# init_alpha=0.001
# tol_obj=1e-12
# tol_grad=1e-08
# tol_param=1e-08
# tol_rel_obj=10000
# tol_rel_grad=1e+07
# history_size=5
# sample_file=test.txt
# append_samples=0
#

Expected Output:

Output of the case without explicitly setting save_iterations should match the one with save_iterations = FALSE as per documentation, and using save_iterations = TRUE should return the saved iterations.

RStan Version:

2.32.6

R Version:

4.3.0

Operating System:

Windows 10

Start at the optimizing() entry point and the documentation for its save_iterations argument, then run the reproducible example from the issue on RStan 2.32.6. Check both console behavior and the generated output file for FALSE, TRUE, and the default; done means the default matches the documented value and TRUE actually saves iterations.