Nested `max(int, int)` expressions cause compiler error in RStan / StanHeaders 2.32.5

Summary:

Compilation fails with rstan 2.32.5 and StanHeaders 2.32.5 under R 4.3.x if the model code contains an expression of the form max(max(int, int), int), i.e. nested calls to max() with integer arguments.

Description:

After recently updating from rstan 2.32.2 and StanHeaders 2.26.28 to the current 2.32.5 versions of both packages in R 4.3.2, I was unable to compile some previously working Stan models. Through a lot of trial and error, removing code blocks until compilation succeeded and then adding them back until it failed again, I managed to isolate the issue to nested integer-max calls. If a model contains an expression like max(max(int, int), int), the compiler throws an error. If the inner max(int, int) is replaced by a constant integer or a previously defined int variable, or if the arguments to max() are real instead of int, compilation works.

A colleague and I have reproduced this in R 4.3.0 and 4.3.2 on two machines running different versions of Windows.

Reproducible Steps:

test1.stan <- "transformed data {
  int test = max(max(1, 1), 1);
}
"

test1 <- rstan::stan_model(model_code = test1.stan)  # error

test2.stan <- "transformed data {
  int test = max(1, 1);
}
"

test2 <- rstan::stan_model(model_code = test2.stan)  # no error

Current Output:

The first example produces the following compiler output followed by an error:

make cmd is
  make -f "C:/R/R-43~1.2/etc/x64/Makeconf" -f "C:/R/R-43~1.2/share/make/winshlib.mk" -f "C:/~/.R/Makevars.win" CXX='$(CXX17) $(CXX17STD)' CXXFLAGS='$(CXX17FLAGS)' CXXPICFLAGS='$(CXX17PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX17LDFLAGS)' SHLIB_LD='$(SHLIB_CXX17LD)' SHLIB="file7fc28f11320.dll" WIN=64 TCLBIN= OBJECTS="file7fc28f11320.o"

make would use
g++  -std=gnu++17 -I"C:/R/R-43~1.2/include" -DNDEBUG   -I"C:/R/R-4.3.2/library/Rcpp/include/"  -I"C:/R/R-4.3.2/library/RcppEigen/include/"  -I"C:/R/R-4.3.2/library/RcppEigen/include/unsupported"  -I"C:/R/R-4.3.2/library/BH/include" -I"C:/R/R-4.3.2/library/StanHeaders/include/src/"  -I"C:/R/R-4.3.2/library/StanHeaders/include/"  -I"C:/R/R-4.3.2/library/RcppParallel/include/" -DRCPP_PARALLEL_USE_TBB=1 -I"C:/R/R-4.3.2/library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DUSE_STANC3 -DSTRICT_R_HEADERS  -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION  -D_HAS_AUTO_PTR_ETC=0  -include "C:/R/R-4.3.2/library/StanHeaders/include/stan/math/prim/fun/Eigen.hpp"  -std=c++1y    -I"C:/rtools43/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c file7fc28f11320.cpp -o file7fc28f11320.o
if test "zfile7fc28f11320.o" != "z"; then \
  if test -e "file7fc28f11320-win.def"; then \
    echo g++  -shared -s -static-libgcc -o file7fc28f11320.dll file7fc28f11320-win.def file7fc28f11320.o  "C:/R/R-4.3.2/library/rstan/lib/x64/libStanServices.a" -L"C:/R/R-4.3.2/library/StanHeaders/libs/x64" -lStanHeaders -L"C:/R/R-4.3.2/library/RcppParallel/lib/x64" -ltbb -LC:/R/R-4.3.2/library/RcppParallel/lib/x64 -ltbb -ltbbmalloc -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/R/R-43~1.2/bin/x64" -lR ; \
    g++  -shared -s -static-libgcc -o file7fc28f11320.dll file7fc28f11320-win.def file7fc28f11320.o  "C:/R/R-4.3.2/library/rstan/lib/x64/libStanServices.a" -L"C:/R/R-4.3.2/library/StanHeaders/libs/x64" -lStanHeaders -L"C:/R/R-4.3.2/library/RcppParallel/lib/x64" -ltbb -LC:/R/R-4.3.2/library/RcppParallel/lib/x64 -ltbb -ltbbmalloc -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/R/R-43~1.2/bin/x64" -lR ; \
  else \
    echo EXPORTS > tmp.def; \
    nm file7fc28f11320.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \
    echo g++  -shared -s -static-libgcc -o file7fc28f11320.dll tmp.def file7fc28f11320.o  "C:/R/R-4.3.2/library/rstan/lib/x64/libStanServices.a" -L"C:/R/R-4.3.2/library/StanHeaders/libs/x64" -lStanHeaders -L"C:/R/R-4.3.2/library/RcppParallel/lib/x64" -ltbb -LC:/R/R-4.3.2/library/RcppParallel/lib/x64 -ltbb -ltbbmalloc -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/R/R-43~1.2/bin/x64" -lR ; \
    g++  -shared -s -static-libgcc -o file7fc28f11320.dll tmp.def file7fc28f11320.o  "C:/R/R-4.3.2/library/rstan/lib/x64/libStanServices.a" -L"C:/R/R-4.3.2/library/StanHeaders/libs/x64" -lStanHeaders -L"C:/R/R-4.3.2/library/RcppParallel/lib/x64" -ltbb -LC:/R/R-4.3.2/library/RcppParallel/lib/x64 -ltbb -ltbbmalloc -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/R/R-43~1.2/bin/x64" -lR ; \
    rm -f tmp.def; \
  fi \
fi

Error in compileCode(f, code, language = language, verbose = verbose) :
C:/R/R-4.3.2/library/RcppEigen/include/Eigen/src/Core/ProductEvaluators.h:35:90: required from 'Eigen::internal::evaluator<Eigen::Product<Lhs, Rhs, Option> >::evaluator(const XprType&) [with Lhs = Eigen::Product<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double, double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1> >, const Eigen::Transpose<Eigen::Matrix<double, -1, 1> > >, Eigen::Matrix<double, -1, -1>, 0>; Rhs = Eigen::Matrix<double, -1, 1>; int Options = 0; XprType = Eigen::Product<Eigen::Product<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double, double>, const Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, const Eigen::Matrix<double, 1, -1> >, const Eigen::Transpose<Eigen::Matrix<double, -1, 1> > >, Eigen::Matrix<double, -1, -1>, 0>, Eigen::Matrix<double, -1, 1>, 0>]'C:/R/R-4.3.2/library/RcppEigen/include/Eigen/src/Core/Product.h:132:22: required from 'Eigen::in

Error in sink(type = "output") : invalid connection

Expected Output:

Successful compilation producing a stanmodel object.

RStan Version:

rstan 2.32.5 and StanHeaders 2.32.5

R Version:

R 4.3.0 or 4.3.2

Operating System:

Windows

Run the two examples using rstan::stan_model with test1.stan and test2.stan under the reported rstan and StanHeaders versions. Compare the nested integer max case with the working cases and trace the resulting compilation path. Done means the nested expression compiles successfully and produces a stanmodel object.