New option for hotstart to re-evaluate function if the fail flag is set to True

Type of issue

What types of issue is it?
Select the appropriate type(s) that describe this issue

• New feature (non-breaking change which adds functionality)

Description

We should add a new option for the hotstart, to better handle failed evaluations.

Current behavior

During hotstart, if any evaluations failed, the same fail flag will be passed to the optimizer, creating a consistent hotstart "replay".

Expected behavior

Add a new option, so that at the first instance of a failed evaluation, an actual evaluation is done. This would be useful if the user has adjusted some solver settings, in order to prevent the original failure. This would of course break the hotstart and any evaluations after that would be actual evaluations not from the history file. So ideally, this would be useful if the only failure happened near the end of the optimization. Eventually, we may want to provide more granular controls over exactly which fail flag we want to re-evaluate during the hotstart, via callCounter indexing.

Start by locating the hotstart implementation and the handling of failed evaluations and the history file. Trace how the fail flag reaches the optimizer, then determine where a new option can trigger an actual evaluation on the first failure. Done means the option preserves replay by default and switches subsequent evaluations to actual runs after that failure.