Rstan 2.21.2 crashes or stops with "Selection" prompt

Hi Stan developers,

Thanks for building such a useful and sophisticated sampling tool. I've been wrestling with what looks to me like a bug today (although I could well be doing something wrong).

Edit: The error report below is for a single model, but I am experiencing this issue for a huge range of models, including a simple AR(1), and on both R 4.1.0 and R 3.6.1. I'm happy to provide more information if it would help.

Edit 2 Jul/29/21: Other things I have tried: removing optimization switches in Makevars, removing Makevars entirely, disabling renv, wiping package libraries and reinstalling packages from scratch. I tried removing R 4.1 and installing R 3.6.1 from the OS package manager, then reinstalling R 4.1. I'm running out of ideas here.

Summary:

rstan (Version 2.21.2, GitRev: 2e1f913d3ca3) crashes or stops with "Selection" prompt when trying to run the CAR case study on Ubuntu 20.04 and R 4.1.0.

Edit: some very simple models are also affected, including basic glms.

Description:

I have been trying to reproduce Max Joseph's case study (source) that fits a CAR model, part of the Stan documentation. I am using the "sparse" parameterization of the model.

I am intermittently getting a range of different outcomes when I try to run the code. When run as a non-interactive script:

• The OS kills the process because it tries to allocate too much memory (my system has 32 GB physical memory and nothing else is running, so this is surprising)
• Segfault

When run interactively, I see an obtuse prompt asking for a "Selection".

Reproducible Steps:

Minimal reproduction script, which is just a simplified version of the original case study, is available here: https://github.com/kuperov/stan_strange_crash

I am running the code non-interactively using Rscript and interactively using the ordinary R repl.

Current Output:

The linked repro script intermittently results in one of 3 outcomes.

Outcome 1, which occurs about 50% of the time when run non-interactively, shows the following:

Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)

SAMPLING FOR MODEL 'car_sparse' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 7.5e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.75 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Chain 1: Iteration:    1 / 10000 [  0%]  (Warmup)
Killed

and tail /var/log/syslog shows the reason: the OOM reaper killed the process because it tried to allocate too much memory:

Jul 16 16:17:50 lump kernel: [27591.327653] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service,task=R,pid=44443,uid=1000
Jul 16 16:17:50 lump kernel: [27591.327690] Out of memory: Killed process 44443 (R) total-vm:47762496kB, anon-rss:31768312kB, file-rss:44kB, shmem-rss:0kB, UID:1000 pgtables:63392kB oom_score_adj:0
Jul 16 16:17:51 lump kernel: [27591.938075] oom_reaper: reaped process 44443 (R), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

The output for outcome 2 (about 50% of the time when run non-interactively) is:

Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)

SAMPLING FOR MODEL 'car_sparse' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 4.1e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.41 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Segmentation fault (core dumped)

Outcome 3, which occurs when the script is run interactively:

acooper@lump:~/src/stan_strange_crash$ R

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

* Project '~/src/stan_strange_crash' loaded. [renv 0.13.2-102]
> source('inference.R')
Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)

SAMPLING FOR MODEL 'car_sparse' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 4e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.4 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Selection:

Stan blocks at that last prompt, and can be interrupted with ctrl+C. I have no idea what it is asking for.

Expected Output:

Expected stan's usual sampling progress output.

RStan Version:

rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)

R Version:

R version 4.1.0 (2021-05-18) from https://cloud.r-project.org/bin/linux/ubuntu

R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] rstan_2.21.2         ggplot2_3.3.5        StanHeaders_2.21.0-7

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7         pillar_1.6.1       compiler_4.1.0     prettyunits_1.1.1 
 [5] tools_4.1.0        pkgbuild_1.2.0     jsonlite_1.7.2     lifecycle_1.0.0   
 [9] tibble_3.1.2       gtable_0.3.0       pkgconfig_2.0.3    rlang_0.4.11      
[13] cli_3.0.0          parallel_4.1.0     curl_4.3.2         loo_2.4.1         
[17] gridExtra_2.3      withr_2.4.2        vctrs_0.3.8        stats4_4.1.0      
[21] grid_4.1.0         glue_1.4.2         inline_0.3.19      R6_2.5.0          
[25] processx_3.5.2     fansi_0.5.0        callr_3.7.0        magrittr_2.0.1    
[29] codetools_0.2-18   scales_1.1.1       ps_1.6.0           ellipsis_0.3.2    
[33] matrixStats_0.59.0 colorspace_2.0-2   renv_0.13.2-102    V8_3.4.2          
[37] utf8_1.2.1         RcppParallel_5.1.4 munsell_0.5.0      crayon_1.4.1

Operating System:

Ubuntu 20.04.2. gcc 9.3.0.

Start with the linked stan_strange_crash reproduction and its inference.R script, then compare the interactive R and Rscript runs described in the issue. Reproduce the behavior with rstan 2.21.2 on the stated Ubuntu and R versions, and trace why sampling reaches an OOM kill, segfault, or "Selection" prompt. Done means the reproduction produces normal sampling progress without those outcomes.