A formal documentation of Math library

Summary:

Per discussion at https://github.com/stan-dev/cmdstan/issues/630#issuecomment-404915871, a formal documentation of Math library is desirable. This is in addition to the function documented in Stan docs. In particular, the process of makeing an application using Math with external libraries, without and without MPI support.

Description:

So far the Math documentation has been mainly

• doxygen
• https://arxiv.org/abs/1509.07164

As the discussion at https://github.com/stan-dev/cmdstan/issues/630#issuecomment-404873569, the Math library provides identifiable features and can be used outside of Stan suites, such as in optimization problems. Ideally we should document the following

• User-faced functions (as pointed out at https://github.com/stan-dev/cmdstan/issues/630#issuecomment-404915871, this could be shared with Stan).
• Functionalities that provided by included libraries but not user-faced in Math, such as many functions in CVODES.
• Included libraries and dependencies, and related functionaries in Math.
• Common types(std::vector, double, Eigen::Matrix, with var and double versions).
• Unincluded dependencies and how to obtain them. In particular the MPI implementations.
• How to make.
  • How to make a sequential application.
  • How to make an MPI application. How to point to correct MPI implementation during making.
  • How to make multi-thread application.
  • Other flags/rules that affect making.
  • Optional, but since Math is a numerical library: how to make with user's fortran codes.
• How to test.
  • Tested compilers
  • Tested platforms
  • Tested MPI implementations
• How to profile and performance tuning.
• Macros and runtime options.
• Parallel communications. Such as examples of MPI job using mpi_cluster. How it may conflict with user's external MPI-enabled library(e.g., explicit call of MPI_Init, as in many solver libraries, will cause runtime failure).
• Debugging(currently there is none).
• Error handling.
• How the derivatives are calculated and can be incorporated in a general application, e.g. gradient-based optimization.
• Code examples.

Current Output:

n/a

Current Version:

v2.18.0

No files or tests are named. Start by reading the linked CmdStan discussion, then inventory the requested areas: APIs, dependencies, build modes, testing, MPI, profiling, debugging, and examples. Done means producing an agreed formal Math library guide that covers the selected scope and its supported platforms and workflows.