[ML] Better adherence to model_memory_limit when in hard_limit

When a job is in the `hard_limit` state it will not model new entities but will continue to model existing entities. However, the memory usage of the job can still increase significantly if models evolve to more memory intensive forms for the entities that were already being modelled.

We should endeavour to adhere more rigorously to the configured model memory limit. Some options for doing this could be:

1. Do not make major model changes, such as detecting periodicity or multi-modal distribution, after the job is in `hard_limit`
2. Discard already-modelled partitions if model memory increases above the configured model memory limit