Formulation

The current code solves dynamic programs with

• reward function f(s, x) and
• state transition function g(s, x, e)

where

1. s is an N-dimensional continuous state variable
2. x is a 1-dimensional continuous action variable and
3. e is a discrete random variable, whose distribution is state independent.

Discussion:

1. Do we want to allow discrete state variables in some dimensions?
2. We want to allow discrete actions (#5) and multidimensional actions.
3. Do we want to allow state dependent distributions?

Start by reading the current dynamic-programming formulation and the discussion in issue #5. The issue asks for decisions about discrete states, discrete and multidimensional actions, and state-dependent distributions; done means establishing the supported formulation and documenting or implementing the agreed scope.