Memory & speed

Presumably Nevegrad performs excellent on computationally expensive objective functions, because it is good at choosing an informative next iterative. On the other hand, it is sometimes slow for choosing this next iterate, which is an issue for computationally cheap objective functions.

Presumably we spend time in the archiving of past iterates.

Ideas:
- disabling, or strongly reducing, the archive when it's not (or almost not) needed for the algorithm.
- (later) recording the computational cost of the objective function for making that decision.

No files, tests, or entry points are named. Start by profiling the archive of past iterates during selection of the next iterate, then compare its cost for computationally cheap objective functions. The issue suggests reducing or disabling archiving first; the later objective-cost-based decision is explicitly deferred, so the completion boundary needs clarification.

Written by the indexing model from the issue text.