stan-dev / stan-dev/cmdstan

Partial specialization failure on Intel compiler

Open
#625 11 comments 0 reactions 1 assignee View on GitHub

@wds15 is already working on this.

Since Jun 19, 2018.

bug makefile
Dominant language
C++
Stars
238
Forks
106
Avg merge
2h 29m
Merged PRs (30d)
6

Description

Summary:

Attempting to link a model w/ threading on Intel compiler produces a partial specialization error.

Description:

I am attempting to test scaling of threaded map_rect on a Xeon Phi system, with the following Stan model,

functions {
    vector lr(vector mu_s, vector j, real[] r, int[] i) {
        real mu = mu_s[1];
        real s = mu_s[2];
        return [normal_lpdf(r | mu, s)]';
    }
}
data { int n; int m; }
transformed data {
    int nm = n * m;
    real x[m, n];
    for (i in 1:m)
        for (j in 1:n)
            x[i, j] = normal_rng(0.0, 1.0);
}
parameters { vector[2] mu_s; vector[0] j[m]; }
model {
    int i[m, 0];
    target += sum(map_rect(lr, mu_s, j, x, i));
}

I confirmed locally use of multiple CPUs and tried to get it working on a HPC Xeon Phi node, first w/ GCC 5.5.0, which compiles fine but uses only one CPU. I assumed GCC doesn't want to thread for the Xeon Phi, so (top doesn't report CPU usage correctly on Xeon Phi). I tried Intel compilers (icpc version 18.0.2, gcc 5.5.0 compatible), but encounter

stan/lib/stan_math/stan/math/rev/scal/meta/operands_and_partials.hpp(63): error: more than one partial specialization matches the template argument list of class "stan::math::internal::ops_partials_edge<double, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>>"
            "stan::math::internal::ops_partials_edge<double, Eigen::Matrix<stan::math::var, R, C, <expression>, R, C>>"
            "stan::math::internal::ops_partials_edge<ViewElt, Eigen::Matrix<Op, R, C, <expression>, R, C>>"
    internal::ops_partials_edge<double, Op1> edge1_;

full output below.

Reproducible Steps:
~/ $ git clone --recursive https://github.com/stan-dev/cmdstan && cd cmdstan
~/cmdstan $ cat <<EOF > make/local
CXX := icpc
CC := icpc
CFLAGS += -std=c11 -O3 -xMIC-AVX512 -fma -align -finline-functions -DEIGEN_ENABLE_AVX512
CXXFLAGS += -DSTAN_THREADS
EOF
~/cmdstan $ make -B -j24 build examples/bernoulli/Bernoulli
...
~/cmdstan $ make examples/map_rect/mrnorm.stan # model source in description above
Current Output:
--- Linking C++ model ---
icpc -Wall -I . -isystem stan/lib/stan_math/lib/eigen_3.3.3 -isystem stan/lib/stan_math/lib/boost_1.66.0 -isystem stan/lib/stan_math/lib/sundials_3.1.0/include -std=c++1y -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -Wno-unused-function -Wno-uninitialized -I src -isystem stan/src -isystem stan/lib/stan_math/ -DFUSION_MAX_VECTOR_SIZE=12 -Wno-unused-local-typedefs -DEIGEN_NO_DEBUG -DSTAN_THREADS -DNO_FPRINTF_OUTPUT -pipe    -O3 -o examples/map_rect/mrnorm src/cmdstan/main.cpp -include examples/map_rect/mrnorm.hpp stan/lib/stan_math/lib/sundials_3.1.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_3.1.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_3.1.0/lib/libsundials_idas.a
icpc: command line warning #10006: ignoring unknown option '-Wno-unused-local-typedefs'
stan/lib/stan_math/stan/math/rev/scal/meta/operands_and_partials.hpp(63): error: more than one partial specialization matches the template argument list of class "stan::math::internal::ops_partials_edge<double, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>>"
            "stan::math::internal::ops_partials_edge<double, Eigen::Matrix<stan::math::var, R, C, <expression>, R, C>>"
            "stan::math::internal::ops_partials_edge<ViewElt, Eigen::Matrix<Op, R, C, <expression>, R, C>>"
    internal::ops_partials_edge<double, Op1> edge1_;
                                             ^
          detected during:
            instantiation of class "stan::math::operands_and_partials<Op1, Op2, Op3, Op4, Op5, stan::math::var> [with Op1=Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Op2=Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Op3=double, Op4=double, Op5=double]" at line 161 of "/usr/local/software/jurecabooster/Stages/2018a/software/GCCcore/5.5.0/include/c++/5.5.0/bits/stl_vector.h"
            instantiation of "std::_Vector_base<_Tp, _Alloc>::~_Vector_base() [with _Tp=stan::math::operands_and_partials<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, double, double, double, stan::math::var>, _Alloc=std::allocator<stan::math::operands_and_partials<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, double, double, double, stan::math::var>>]" at line 257 of
                      "/usr/local/software/jurecabooster/Stages/2018a/software/GCCcore/5.5.0/include/c++/5.5.0/bits/stl_vector.h"
            instantiation of "std::vector<_Tp, _Alloc>::vector() [with _Tp=stan::math::operands_and_partials<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, double, double, double, stan::math::var>, _Alloc=std::allocator<stan::math::operands_and_partials<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>, double, double, double, stan::math::var>>]" at line 60 of
                      "stan/lib/stan_math/stan/math/prim/mat/functor/map_rect_combine.hpp"
            instantiation of "stan::math::internal::map_rect_combine<F, T_shared_param, T_job_param>::map_rect_combine(const Eigen::Matrix<T_shared_param, -1, 1, 0, -1, 1> &, const std::vector<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>>> &) [with F=mrnorm_model_namespace::lr_functor__, T_shared_param=stan::math::var, T_job_param=stan::math::var]" at line 127 of "stan/lib/stan_math/stan/math/prim/mat/functor/map_rect_concurrent.hpp"
            instantiation of "Eigen::Matrix<stan::return_type<T_shared_param, T_job_param, double, double, double, double>::type, -1, 1, 0, -1, 1> stan::math::internal::map_rect_concurrent<call_id,F,T_shared_param,T_job_param>(const Eigen::Matrix<T_shared_param, -1, 1, 0, -1, 1> &, const std::vector<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>>> &, const std::vector<stan::math::internal::ops_partials_edge<double,
                      std::vector<std::vector<stan::math::var, std::allocator<stan::math::var>>, std::allocator<std::vector<stan::math::var, std::allocator<stan::math::var>>>>>::partial_t, std::allocator<stan::math::internal::ops_partials_edge<double, std::vector<std::vector<stan::math::var, std::allocator<stan::math::var>>, std::allocator<std::vector<stan::math::var, std::allocator<stan::math::var>>>>>::partial_t>> &, const std::vector<std::vector<int, std::allocator<int>>,
                      std::allocator<std::vector<int, std::allocator<int>>>> &, std::ostream *) [with call_id=1, F=mrnorm_model_namespace::lr_functor__, T_shared_param=stan::math::var, T_job_param=stan::math::var]" at line 176 of "stan/lib/stan_math/stan/math/prim/mat/functor/map_rect.hpp"
            instantiation of "Eigen::Matrix<stan::return_type<T_shared_param, T_job_param, double, double, double, double>::type, -1, 1, 0, -1, 1> stan::math::map_rect<call_id,F,T_shared_param,T_job_param>(const Eigen::Matrix<T_shared_param, -1, 1, 0, -1, 1> &, const std::vector<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<T_job_param, -1, 1, 0, -1, 1>>> &, const std::vector<stan::math::internal::ops_partials_edge<double, std::vector<std::vector<stan::math::var,
                      std::allocator<stan::math::var>>, std::allocator<std::vector<stan::math::var, std::allocator<stan::math::var>>>>>::partial_t, std::allocator<stan::math::internal::ops_partials_edge<double, std::vector<std::vector<stan::math::var, std::allocator<stan::math::var>>, std::allocator<std::vector<stan::math::var, std::allocator<stan::math::var>>>>>::partial_t>> &, const std::vector<std::vector<int, std::allocator<int>>, std::allocator<std::vector<int, std::allocator<int>>>> &,
                      std::ostream *) [with call_id=1, F=mrnorm_model_namespace::lr_functor__, T_shared_param=stan::math::var, T_job_param=stan::math::var]" at line 294 of "examples/map_rect/mrnorm.hpp"
            instantiation of "T__ mrnorm_model_namespace::mrnorm_model::log_prob<propto__,jacobian__,T__>(std::vector<T__, std::allocator<T__>> &, std::vector<int, std::allocator<int>> &, std::ostream *) const [with propto__=true, jacobian__=true, T__=stan::math::var]" at line 45 of "stan/src/stan/model/log_prob_grad.hpp"
            instantiation of "double stan::model::log_prob_grad<propto,jacobian_adjust_transform,M>(const M &, std::vector<double, std::allocator<double>> &, std::vector<int, std::allocator<int>> &, std::vector<double, std::allocator<double>> &, std::ostream *) [with propto=true, jacobian_adjust_transform=true, M=stan_model]" at line 149 of "stan/src/stan/services/util/initialize.hpp"
            instantiation of "std::vector<double, std::allocator<double>> stan::services::util::initialize(Model &, stan::io::var_context &, RNG &, double, bool, stan::callbacks::logger &, stan::callbacks::writer &) [with Model=stan_model, RNG=boost::random::ecuyer1988]" at line 56 of "stan/src/stan/services/diagnose/diagnose.hpp"
            instantiation of "int stan::services::diagnose::diagnose(Model &, stan::io::var_context &, unsigned int, unsigned int, double, double, double, stan::callbacks::interrupt &, stan::callbacks::logger &, stan::callbacks::writer &, stan::callbacks::writer &) [with Model=stan_model]" at line 143 of "src/cmdstan/command.hpp"
            instantiation of "int cmdstan::command<Model>(int, const char **) [with Model=stan_model]" at line 8 of "src/cmdstan/main.cpp"

compilation aborted for src/cmdstan/main.cpp (code 2)
make: *** [examples/map_rect/mrnorm] Error 2
Expected Output:

Successful compilation

Additional Information:

I can't provide access to the system but can do some debuggin if any ideas provided.

Current Version:

develop (834df71b2c8f)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.