Add `branch:` key to dynamically generate workflow steps
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 161
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
**Disclaimer/Intro:**
Maestro is powerful, but does not support dynamic workflows like optimization. @jsemler has written a private package called `encorewf` that supports limited iteration. I asked ChatGPT o4-mini for a more general solution, and they (gender-neutral?) suggested the following. [My full chat with ChatGPT](https://chatgpt.com/share/6842d963-4cfc-800b-af10-1563596cda15) is [here](https://chatgpt.com/share/6842d963-4cfc-800b-af10-1563596cda15).
@jwhite242, @jsemler, @FrankD412, @daub1, @antimatterhorn, Please let me know what you think!
**Problem:**
Maestro’s current study spec requires all steps to be defined statically (or via parameter expansion). There is no built-in way to run a script at runtime that emits new step definitions based on completed tasks.
**Proposal:**
Introduce a new `branch:` field under `run:` that executes a user‐supplied command after its `depends:` finish. That command should print valid YAML step definitions to stdout, which Maestro will parse and inject into the DAG on the fly.
**Example:**
```yaml
- name: run-lulesh
run:
cmd: |
$(LULESH)/lulesh2.0 -s $(SIZE) -i $(ITERATIONS) -p > $(outfile)
depends: [make-lulesh]
- name: branch-postprocess
run:
# After all run-lulesh_* instances complete, run this script.
branch: |
python scripts/generate_postprocess_steps.py --input-dir $(OUTPUT_PATH)
depends: [run-lulesh_*]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the proposed scripts/generate_postprocess_steps.py example and the run/depends workflow entry points. Trace how Maestro validates static YAML and builds the DAG, then determine how generated step definitions could be validated and injected after dependencies finish. Done means branch-generated steps execute with the expected dependency behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, yaml
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100