Efficiency problems in min_dim and max_dim

https://github.com/clab/dynet/blob/master/dynet/nodes-minmax.cc

I'm no C++/CUDA/Eigen expert, so here goes:

I ran into some issues with the efficiency of taking max(M, 1) on GPU, where M is a nxm (batched) matrix. This was much slower than taking the mean over the columns, or using MaxPooling2D to do the same thing. Looking at the code, it seems there are two functions: Max, which handles vectors, and MaxDimension, which handles any other dimensional input. MaxDimension runs a double for loop in its gradient calculation regardless of what the input is, which means that our gradient calculation involves O(nm) terms instead of O(m) terms, which doesn't seem right.

Start with dynet/nodes-minmax.cc and inspect the Max and MaxDimension implementations, especially the gradient calculation's double loop. Compare the max(M, 1) GPU path with the mean-over-columns and MaxPooling2D cases described in the issue. Done means preserving the reduction behavior while addressing the reported inefficiency and verifying the affected path with a relevant benchmark or test.

Written by the indexing model from the issue text.