Advice/protection against oddities in training point sets

Arising in PR #265 for example,

We find that sometimes emulator training is problematic for a fixed data set, and a small modification leads to massive improvements. More robust handling of the training dataest by e.g. providing more of a Cross validation procedure, or better construction of train/validation splits in the provided points may lead to more robust trainings.

Start by reviewing PR #265 and the emulator training path that consumes the provided training point sets. Investigate how train/validation splits are currently constructed and whether cross-validation is already supported. Done means defining and implementing a robust handling approach for fixed data sets, with evidence that small changes do not cause large unexplained training differences.

Written by the indexing model from the issue text.