Manually restarting an experiment after job completion
- Dominant language
- Python
- Stars
- 161
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
Suppose I run an experiment that dispatches N jobs, each restarting as many times as specified in restart_limit and restarting using code written in the "restart" section of the YAML script. I would like to be able to also restart the experiment at any point in the future, as opposed to the automatic restart. This way, I can manually restart later on if all of the original restarts fail or if I want to run additional times beyond the specified restart_limit. The restarts would occur within the same experiment folder, as opposed to generating a new folder which occurs when running "maestro run experiment.yaml". Does such a feature exist?
Contributor guide
Research direction
Start by tracing the existing `maestro run experiment.yaml` flow and how the YAML `restart` section and `restart_limit` are handled. Determine how a later manual restart could reuse the existing experiment folder rather than create a new one, and define completion around supporting that workflow after the original jobs finish.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100