Scaling of optimizer with number of parameters

We would like the linear optimizer to scale to a large number of parameters. Generally the more parameters there are in the optimized wavefunction, the more accurate the wavefunction. This issue records some investigation into memory and time scaling with respect to the number of parameters.

Sources of memory usage (Np - number of parameters, Ns - number of samples, Ne - number of electrons)
1. Parameters and associated info in VariableSet - O(Np)
2. Values of psi, and E for each sample - O(Ns), electron positions per sample O(Ns*Ne), derivatives per sample O(Ns*Np)
3. Overlap and Hamiltonian matrices used in solving linear method eigenproblem - O(Np * Np)

The source of memory usage is more detail

**1. Parameters in VariableSet**
Some discussion in #3625

There are often multiple copies of VariableSet, one at the each wavefunction component level. At the top level of the optimizer, there are two copies, for dealing with constraints.

I instrumented the code, and ran on a hydrocarbon molecule with 1-body and 2-body Jastrows, and CI coefficients. The number of coefficients was varied to get the range of total variational parameters. There were 26 instantiations of VariableSet, most of them zero-sized or small (Jastrows). There were 4 large instantiations (CI coeff). The space usage is about 124 bytes/parameter for an individual variable set, and 500 bytes/parameter considering all the variable sets.

(Technical aside: Many the data structures in VariableSet use the parameter name in a std::string. The parameter names were all short enough (<16 characters) to use the short-string optimization. The short string fits in the std::string itself, and there is no additional dynamic allocation. If the names were longer than 15 characters, we would need to measure the allocated string space as well.)

**2. Sample related storage**
I have run into this memory limit with small systems (fast to generate samples) and large sample numbers (trying to be accurate for testing).

There are two variables in QMCCostFunctionBatched of size Ns x Np: `DerivRecords` and `HDerivRecords`

The solution will be to collect the accumulated derivative quantities during the run. The eases the memory pressures (would be O(Np)). For correlated sampling, the samples need to be saved - O(Ns*Ne) and psi and the energy saved per sample O(Ns).

**3. Overlap and Hamiltonian matrices**
In `one_shift_run` there are four matrices of size Np x Np: `ovlMat`, `hamMat`, `invMat` and `prdMat`

Just to make some concrete numbers for a sense of scale, assume the number of samples is 10000 and vary the number of parameters

Np | VariableSet | sample-related storage | linear method matrices
----|----|----|----
10k | .005 GB | 1.6 GB | 3.2 GB
50k | .025 GB | 8.0 GB | 80 GB
100k | .05 GB | 16 GB | 320 GB
1m | .5 GB | 160 GB | 32000 GB

**Scaling in time**

As for scaling in time, solving the generalized eigenvalue problem is O(Np^3), and will grow to dominate the time fairly quickly.
I have created a kernel for this for further exploration at
https://github.com/markdewing/qmc_kernels/tree/master/kernels/generalized_eigensolve